-- Logs begin at Wed 2022-10-05 13:31:38 UTC, end at Wed 2022-10-05 15:08:18 UTC. -- Oct 05 14:02:34.298278 dv-cinder-835878-4-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 05 14:02:34.307133 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 5 14:02:34 2022] *** Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: nodename: dv-cinder-835878-4-iscsi Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: machine: x86_64 Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: clock source: unix Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: pcre jit disabled Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: detected number of CPU cores: 4 Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: current working directory: / Oct 05 14:02:34.309136 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: detected binary path: /usr/bin/uwsgi-core Oct 05 14:02:34.309914 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: your processes number limit is 47680 Oct 05 14:02:34.309914 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: your memory page size is 4096 bytes Oct 05 14:02:34.309914 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: detected max file descriptor number: 2048 Oct 05 14:02:34.309914 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: lock engine: pthread robust mutexes Oct 05 14:02:34.309914 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: thunder lock: enabled Oct 05 14:02:34.309914 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 05 14:02:34.309914 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 05 14:02:34.348076 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: Python main interpreter initialized at 0x563a0b98e9c0 Oct 05 14:02:34.348076 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: python threads support enabled Oct 05 14:02:34.348076 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: your server socket listen backlog is limited to 100 connections Oct 05 14:02:34.348076 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: your mercy for graceful operations on workers is 90 seconds Oct 05 14:02:34.348419 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: mapped 403077 bytes (393 KB) for 2 cores Oct 05 14:02:34.348495 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: *** Operational MODE: preforking *** Oct 05 14:02:34.348740 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: *** uWSGI is running in multiple interpreter mode *** Oct 05 14:02:34.348740 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: spawned uWSGI master process (pid: 105577) Oct 05 14:02:34.348940 dv-cinder-835878-4-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 05 14:02:34.350008 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: spawned uWSGI worker 1 (pid: 105578, cores: 1) Oct 05 14:02:34.350008 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: spawned uWSGI worker 2 (pid: 105579, cores: 1) Oct 05 14:02:34.350008 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 05 14:02:34.997002 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 05 14:02:34.997664 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 05 14:02:36.072467 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:02:36.072467 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: from cryptography.utils import int_from_bytes Oct 05 14:02:36.073222 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:02:36.073222 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: from cryptography.utils import int_from_bytes Oct 05 14:02:36.073222 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:02:36.073222 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: from cryptography.utils import int_from_bytes Oct 05 14:02:36.073776 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:02:36.073776 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: from cryptography.utils import int_from_bytes Oct 05 14:02:38.524408 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 14:02:38.524899 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 14:02:38.525286 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 14:02:38.525437 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 14:02:38.525522 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 14:02:38.525588 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 14:02:38.525693 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 14:02:38.525774 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 14:02:38.525858 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 14:02:38.525945 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 14:02:38.526167 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.526307 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.526448 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.526539 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.526787 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.526927 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.527098 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.527258 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.527418 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.527512 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.527733 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.527841 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.528042 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.528134 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.528355 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.528452 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.528678 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.528770 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.528933 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.529043 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.529243 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.529335 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.529587 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835878-4-iscsi {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.529720 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835878-4-iscsi {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.529935 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.530035 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.530236 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.530322 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.530546 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.530741 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.530894 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.531049 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.531187 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.531326 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.531461 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.531609 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.531743 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.531884 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.532209 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.532383 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.533430 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.533587 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.533587 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.533915 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.534050 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.534189 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.534322 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.534464 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.534610 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.534764 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.534896 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535028 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535159 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535298 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535434 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535571 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535705 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535848 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.535981 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.536122 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.536266 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835878-4-iscsi {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.536413 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835878-4-iscsi {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.536548 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.536694 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.536930 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.537089 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.537252 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.537460 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.537616 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.537803 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.537942 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.538123 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.538260 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.538410 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.538544 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.538676 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.538819 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.538964 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.539100 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.539240 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.539385 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.539570 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.539716 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.539851 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.539983 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.540126 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.540264 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.540397 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.540542 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.540675 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.540820 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.540953 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.541095 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.541247 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.541387 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.541526 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.541690 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.541829 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.541985 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.542128 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.542261 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.542399 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.542532 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.542663 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.542799 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.542945 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.543078 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.543210 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.543352 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.543489 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.543644 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.543780 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.543919 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.544061 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.544249 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.544388 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.544535 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.544679 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.544813 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.544945 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.545078 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.545382 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.545528 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.545706 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.545845 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.545978 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.546111 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.546257 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.546394 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.546535 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.546668 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.546806 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.546972 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.547128 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.547273 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.547405 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.547619 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.547765 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.547913 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.548048 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.548186 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.548325 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.548457 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.548588 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.548729 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.548863 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.549011 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.549151 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.549291 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.549424 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.549566 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.549735 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.549871 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550003 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550135 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550278 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550420 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550553 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550825 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.550964 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.551108 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.551249 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.551382 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.551513 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.551650 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.551783 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.551923 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.552063 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.552197 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.552340 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.552473 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.552614 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.552755 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.552889 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.553026 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.553164 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.553298 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.553428 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.553568 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.553737 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.553880 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.554013 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.554154 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.554288 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.554443 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.554578 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.554718 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.554852 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.555102 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.555258 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.555394 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.555527 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.555658 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.555798 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.555930 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.556077 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.556210 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.556362 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.556501 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.556634 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.556765 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.556905 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.557052 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.557201 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.557334 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.557467 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.557623 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.557890 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.558033 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.558178 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.558311 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.558450 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.558598 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.558738 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.558878 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.559026 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.559174 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.559307 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.559445 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.559588 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.559727 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.559866 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560008 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560141 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560272 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560412 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560546 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560686 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560827 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.560965 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.561103 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.561248 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.561382 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.561514 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.561681 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.561831 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.561977 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.562111 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.562243 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.562375 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.562506 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.562640 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.562824 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.562971 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.563138 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.563283 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.563417 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.563550 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.563702 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.563837 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.564010 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.564156 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.564298 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.564429 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.564570 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.564703 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.564909 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.565108 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.565256 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.565407 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.565551 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.565774 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.565774 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566033 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566165 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566307 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566440 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566571 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566711 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566848 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.566979 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.567127 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.567267 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.567416 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.567549 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:02:38.567681 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.567926 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.568087 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.568222 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.568362 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.568495 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.568625 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.568807 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.568957 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.569159 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.569359 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.569521 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.569713 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.569872 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.570018 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.570152 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.570284 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.570416 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.570547 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.570763 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.570907 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.571041 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.571173 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.571323 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.571465 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.571598 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.571730 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.571882 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.572027 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.572175 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.572316 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.572457 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.572599 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.572738 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.572870 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.573088 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.573314 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.573478 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.573618 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.573816 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.573962 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.574097 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.574241 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.574382 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.574515 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.574647 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.574785 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.574988 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.575202 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.575365 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.575504 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.575638 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.575769 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.575918 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.576065 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.576234 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.576491 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.576646 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.576781 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.576927 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.577071 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.577216 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.577360 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.577493 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.577624 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.577820 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.577981 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.578196 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.578335 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.578478 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.578612 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.578753 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.578895 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.579029 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.579171 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.579302 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.579435 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.579613 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.579754 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.579921 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.580068 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.580227 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.580368 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.580521 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.580657 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.580834 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.580972 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.581146 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.581293 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.581466 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.581606 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.581793 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.581940 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.582084 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.582230 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.582396 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.582545 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.582706 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.582845 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.583014 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.583172 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.583313 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.583453 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.583607 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.583745 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.583909 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.584056 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.584189 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.584329 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.584499 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.584637 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.584800 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.584949 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.585092 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.585223 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.585406 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.585545 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.585728 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.585876 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.586054 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.586192 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.586354 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.586491 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.586662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.586805 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.586970 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.587109 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.587275 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.587421 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.587564 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.587704 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.587873 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.588018 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.588246 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.588402 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.588545 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.588686 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.588819 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.588962 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.589095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.589241 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.589375 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.589505 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.589688 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.589839 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.589981 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.590131 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.590273 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.590407 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.590563 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.590703 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.590872 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.591029 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.591163 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.591300 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.591472 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.591611 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.591781 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.591930 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.592111 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.592253 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.592421 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.592568 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.592796 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.592932 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.593065 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.593203 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.593358 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.593498 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.593666 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.593807 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.593987 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.594127 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.594310 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.594449 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.594623 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.594782 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.594917 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.595110 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.595250 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.595392 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.595538 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.595672 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.595803 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.595940 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.596074 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.596216 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.596357 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.596490 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.596653 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.596793 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.596954 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.597102 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.597258 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.597398 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.597565 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.597753 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.597905 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.598055 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.598189 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.598349 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.598552 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.598699 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.598832 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.598974 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.599115 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.599248 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.599389 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.599522 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.599663 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.599795 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.599934 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.600067 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.600251 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.600407 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.600587 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.600726 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.600903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.601050 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.601216 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.601354 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.601522 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.601687 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.601857 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.601995 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.602164 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.602302 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.602456 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.602590 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.602761 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.602899 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.603067 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.603213 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.603373 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.603511 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.603678 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.603815 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.603985 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.604131 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.604287 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.604434 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.604593 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.604738 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.604897 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.605050 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.605209 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.605726 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.605864 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.605995 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.606174 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.606322 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.606486 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.606625 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.606823 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.606971 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.607133 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.607278 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.607452 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.607591 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.607809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.607950 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.608122 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.608262 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.608425 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.608579 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.608794 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.608943 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.609160 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.609308 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.609441 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.609579 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.609792 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.609928 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.609928 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.610192 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.610323 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.610462 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.610595 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.610726 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.610885 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.611026 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.611165 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.611313 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.611452 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.611597 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.611736 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.611881 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.612021 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.612160 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.612305 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.612453 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.612608 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.612755 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.612893 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.613033 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.613170 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.613316 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.613493 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.613664 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.613810 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.613948 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.614117 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.614274 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.614415 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.614564 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.614703 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.614841 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.614998 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.615138 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.615277 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.615414 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.615587 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.615732 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.615882 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.616029 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.616190 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.616336 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.616494 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.616641 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.616788 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.616926 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.617085 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.617237 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.617377 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.617516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.617700 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.617857 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.617998 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.618135 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.618275 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.618441 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.618581 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.618726 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.618955 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.619101 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.619240 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.619389 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.619529 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.619666 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.619803 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.619949 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.620091 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.620237 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.620395 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.620542 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.620682 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.620819 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.620957 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.621181 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.621323 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.621462 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.621607 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.621818 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.621959 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.622114 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.622265 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.622403 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.622542 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.622679 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.622826 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.622973 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.623120 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.623280 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.623425 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.623565 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.623712 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.623852 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.623991 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.624140 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.624288 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.624427 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.624575 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.624714 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.624852 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.624989 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.625127 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.625266 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.625439 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.625588 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.625768 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.625918 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.626067 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.626215 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.626355 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.626501 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.626648 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.626794 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.626933 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.627079 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.627218 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.627357 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.627526 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.627672 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.627823 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.627980 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.628128 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.628245 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.628521 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.628708 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.629054 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.629217 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.629343 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.629495 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.629668 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.629828 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.630006 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.630169 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.630319 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.630467 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.630617 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.630765 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.630944 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.631089 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.631255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.631410 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.631557 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.631704 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.631855 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.631996 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.632144 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.632294 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.632442 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.632582 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.632733 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.632873 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.633021 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.633161 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.633299 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.633437 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.633575 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.633742 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.633898 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.634056 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.634196 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.634335 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.634479 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.634625 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.634773 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.634921 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.635061 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.635207 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.635346 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.635484 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.635638 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.635789 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.635935 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.636075 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.636214 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.636360 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.636500 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.636639 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.636807 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.636961 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.637104 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.637251 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.637390 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.637535 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.637750 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.637896 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.638044 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.638193 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.638333 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.638478 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.638641 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.638788 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.639087 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.639248 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.639461 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.639585 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.639745 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.639849 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.640045 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.640137 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.640358 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.640498 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.640635 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.640734 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.640953 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.641087 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.641248 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.641382 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.641532 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.641625 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.641857 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.641948 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.642159 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.642255 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.642461 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.642561 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.642764 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.642860 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.643068 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.643159 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.643372 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.643477 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.643669 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.643761 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.643954 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.644041 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.644250 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.644420 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.644550 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.644635 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.644852 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.644944 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.645155 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.645254 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.645448 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.645554 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.645767 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.645855 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.646070 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.646157 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.646367 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.646453 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.646668 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.646753 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.646965 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.647058 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.647269 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.647362 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.647571 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.647657 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.647869 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.647957 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.648166 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.648331 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.648467 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.648561 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.648767 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.648865 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.649063 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.649149 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.649363 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.649461 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.649687 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.649776 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.649988 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.650087 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.650287 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.650384 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.650588 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.650682 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.650884 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.650976 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.651182 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.651277 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.651481 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.651622 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.651790 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.651877 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.652098 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.652197 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.652397 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.652484 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.652700 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.652785 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.652995 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.653088 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.653295 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.653381 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.653594 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.653704 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.653922 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.654096 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.654255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.654398 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.654558 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.654702 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.654851 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.654992 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.655132 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.655281 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.655429 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.655575 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.655729 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.655872 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.656012 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.656160 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.656299 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.656448 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.656588 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.656726 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.656864 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.657003 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.657153 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.657301 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.657441 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.657587 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.657807 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.657961 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.658111 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.658258 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.658400 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.658547 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.658711 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.658856 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.659020 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.659189 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.659412 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.659566 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.659742 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.659883 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.660021 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.660177 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.660318 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.660464 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.660603 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.660755 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.660903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.661043 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.661191 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.661331 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.661470 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.661608 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.661790 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.661932 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.662079 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.662218 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.662358 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.662497 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.662649 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.662788 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.662937 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.663077 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.663216 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.663355 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.663501 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.663644 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.663781 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.663932 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.664079 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.664227 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.664367 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.664506 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.664644 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.664792 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.665089 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.665253 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.665411 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.665559 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.665729 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.665872 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.666011 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.666160 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.666309 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.666449 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.666589 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.666727 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.666877 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.667024 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.667171 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.667311 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.667449 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.667589 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.667736 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.667877 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.668016 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.668162 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.668301 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.668440 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.668605 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.668748 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.668887 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.669025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.669162 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.669310 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.669539 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.669700 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.669846 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.669985 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.670143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.670314 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.670471 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.670623 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.670764 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.670902 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.671041 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.671187 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.671327 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.671465 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.671603 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.671742 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.671888 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.672105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.672248 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.672393 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.672545 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.672692 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.672832 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.672980 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.673120 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.673306 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.673453 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.673619 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.673797 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.673949 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.674097 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.674237 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.674382 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.674532 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.674684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.674824 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.674964 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.675109 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.675249 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.675433 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.675588 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.675728 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.675866 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.676029 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.676175 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.676324 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.676478 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.676619 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.676765 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.676930 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.677085 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.677254 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.677413 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.677653 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.677802 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.677992 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.678138 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.678294 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.678444 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.678649 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.678796 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.678968 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.679136 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.679288 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.679437 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.679665 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.679812 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.679952 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.680090 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.680229 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.680368 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.680525 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.680666 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.680806 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.680944 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.681089 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.681230 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.681381 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.681522 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.681688 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.681844 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.681985 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.682124 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.682338 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.682494 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.682633 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.682774 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.682948 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.683101 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.683251 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.683393 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.683540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.683686 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.683826 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.683974 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.684168 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.684325 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.684488 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.684634 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.684810 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.684965 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.685105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.685245 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.685384 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.685523 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.685696 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.685890 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.686039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.686181 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.686338 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.686479 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.686639 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.686779 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.686926 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.687074 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.687214 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.687352 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.687500 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.687640 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.687778 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.687917 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.688063 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.688219 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.688367 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.688515 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.688654 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.688792 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.688936 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.689078 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.689226 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.689366 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.689539 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.689782 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.689930 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.690089 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.690230 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.690368 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.690506 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.690647 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.690785 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.690933 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.691072 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.691211 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.691348 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.691488 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.691628 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.691775 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.691934 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.692074 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.692235 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.692381 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.692541 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.692696 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.692844 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.692985 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.693170 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.693315 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.693492 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.693704 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.693851 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.694012 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.694152 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.694299 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.694447 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.694587 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.694752 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.694897 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.695044 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.695185 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.695360 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.695506 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.695683 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.695846 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696044 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696192 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696312 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696406 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696595 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696727 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696871 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.696982 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.697195 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.697331 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.697480 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.697588 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.697831 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.697988 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.698113 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.698228 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.698444 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.698578 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.698743 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.698888 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.699054 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.699187 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.699345 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.699476 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.699665 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.699804 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.699966 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.700083 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.700237 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.700394 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.700545 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.700687 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.700828 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.700967 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.701120 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.701265 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.701599 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.701805 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.701958 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.702102 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.702243 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.702395 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.702545 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.702686 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.702826 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.702965 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.703116 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.703257 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.703409 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.703557 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.703714 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.703854 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.703993 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.704144 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.704292 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.704433 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.704605 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.704752 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.704913 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.705072 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.705219 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.705367 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.705507 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.705678 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.705853 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.706009 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.706150 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.706370 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.706517 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.706700 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.706978 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.707133 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.707308 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.707452 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.707612 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.707757 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.707919 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.708067 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.708206 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.708376 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.708522 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.708663 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.708813 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.708954 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.709096 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.709289 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.709446 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.709589 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.709792 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.709935 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.710156 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.710302 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.710450 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.710669 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.710811 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.710964 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.711106 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.711253 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.711394 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.711542 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.711695 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.711836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.711975 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.712115 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.712255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.712409 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.712557 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.712698 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.712837 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.712977 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.713128 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.713278 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.713417 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.713556 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.713717 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.713871 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.714012 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.714152 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.714297 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.714437 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.714577 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.714731 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.714871 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.715012 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.715170 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.715311 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.715450 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.715599 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.715749 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.715890 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.716029 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.716195 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.716336 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.716485 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.716655 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.716808 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.716968 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.717134 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.717284 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.717424 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.717598 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.717767 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.717932 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.718079 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.718385 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.718540 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.718703 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.718849 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.719056 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.719215 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.719365 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.719509 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.719689 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.719835 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.719985 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.720151 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.720359 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.720511 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.720660 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.720799 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.720938 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.721097 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.721238 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.721378 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.721524 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.721707 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.721853 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.722004 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.722145 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.722285 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.722424 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.722570 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.722717 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.722870 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.723025 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.723166 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.723305 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.723443 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.723594 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.723734 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.724051 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.724196 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.724345 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.724484 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.724688 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.724879 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.725025 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.725222 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.725368 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.725540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.725741 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.725895 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.726035 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.726210 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.726356 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.726521 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.726675 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.726815 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.726961 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.727122 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.727290 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.727446 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.727595 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.727736 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.727876 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.728016 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.728166 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.728314 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.728455 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.728595 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.728734 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.728872 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.729021 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.729169 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.729309 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.729451 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.729590 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.729799 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.729942 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.730094 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.730235 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.730789 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.730934 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.731074 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.731226 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.731377 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.731526 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.731672 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.731819 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.731960 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.732108 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.732256 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.732396 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.732536 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.732725 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.732872 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.733022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.733175 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.733327 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.733467 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.733607 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.733792 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.733924 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.734011 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.734091 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.734285 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.734429 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.734564 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.734679 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.734891 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.735037 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.735213 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.735346 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.735490 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.735589 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.735808 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.735940 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.736121 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.736254 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.736433 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.736601 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.736686 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.736765 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.737042 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.737176 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.737348 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.737479 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.737657 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.737755 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.738000 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.738153 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.738289 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.738534 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.738667 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.738845 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.739000 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.739239 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.739450 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.739598 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.739771 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.739925 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.740096 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.740253 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.740394 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.740541 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.740691 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.740911 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.741065 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.741214 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.741375 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.741515 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.741672 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.741818 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.741975 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.742122 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.742263 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.742402 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.742541 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.742688 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.742837 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.742985 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.743137 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.743286 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.743433 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.743574 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.743721 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.743868 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.744015 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.744156 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.744307 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.744456 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.744597 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.744762 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.744904 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.745054 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.745194 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.745344 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.745483 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.745623 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.745826 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.745967 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.746106 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.746255 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.746395 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.746534 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.746674 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.746859 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.747010 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.747158 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.747299 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.747439 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.747579 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.747718 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.747871 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.748012 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.748152 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.748318 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.748461 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.748608 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.748771 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.748914 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.749054 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.749192 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.749341 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.749494 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.749831 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.749981 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.750148 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.750294 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.750446 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.750595 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.750956 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.751111 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.751252 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.751400 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.751541 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.751693 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.751834 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.752004 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.752150 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.752297 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.752448 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.752620 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.752776 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.752925 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.753066 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.753237 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.753397 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.753560 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.753742 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.753897 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.754041 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.754215 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.754378 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.754563 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.754707 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.754859 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.754999 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.755146 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.755286 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.755462 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.755608 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.755771 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.755916 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.756063 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.756204 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.756342 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.756481 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.756631 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.756780 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.756927 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.757068 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.757207 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.757346 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.757512 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.757679 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.757836 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.757978 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.758124 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.758315 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.758465 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.758628 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.758771 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.758919 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.759059 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.759199 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.759345 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.759494 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.759641 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.759782 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.759921 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.760060 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.760199 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.760347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.760496 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.760636 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.760776 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.760922 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.761141 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.761296 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.761445 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.761586 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.761756 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.761908 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.762057 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.762199 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.762346 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.762497 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.762637 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.762777 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.762924 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.763067 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.763206 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.763345 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.763494 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.763634 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.763781 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.764017 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.764173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.764334 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.764475 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.764614 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.764767 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.764908 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.765057 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.765198 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.765337 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.765484 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.765624 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.765807 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.765959 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.766101 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.766259 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.766409 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.766549 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.766688 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.766828 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.766968 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.767107 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.767321 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.767464 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.767603 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.767743 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.767883 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.768040 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.768191 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.768339 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.768480 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.768627 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.768768 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.768915 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.769063 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.769204 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.769343 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.769506 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.769686 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.769845 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.770004 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.770144 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.770284 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.770446 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.770594 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.770742 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.770883 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.771038 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.771260 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.771416 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.771567 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.771716 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.771870 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.772011 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.772150 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.772289 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.772439 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.772580 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.772727 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.772868 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.773016 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.773156 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.773304 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.773446 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.773595 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.773795 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.773936 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.774088 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.774229 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.774376 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.774523 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.774690 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.774847 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.774997 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.775144 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.775325 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.775471 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.775649 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.775795 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.775950 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.776099 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.776239 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.776379 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.776541 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.776697 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.776844 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 14:02:38.776986 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.777255 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.777568 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.777934 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:02:38.778175 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105579) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 14:02:38.932500 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO dbcounter [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Registered counter for database nova_api Oct 05 14:02:38.933023 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO dbcounter [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Registered counter for database nova_api Oct 05 14:02:38.954858 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_db.sqlalchemy.engines [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105578) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:38.956425 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_db.sqlalchemy.engines [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105579) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:38.963002 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writer thread running {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:38.963143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writer thread running {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:39.020162 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.context [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),003460f2-e2d0-447e-9352-d4e433e26efc(cell1) {{(pid=105578) load_cells /opt/stack/nova/nova/context.py:464}} Oct 05 14:02:39.023812 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.context [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),003460f2-e2d0-447e-9352-d4e433e26efc(cell1) {{(pid=105579) load_cells /opt/stack/nova/nova/context.py:464}} Oct 05 14:02:39.024033 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_concurrency.lockutils [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105578) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:02:39.024493 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_concurrency.lockutils [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105578) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:02:39.026067 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_concurrency.lockutils [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105578) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:02:39.026771 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_concurrency.lockutils [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105578) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:02:39.027201 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_concurrency.lockutils [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105578) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:02:39.027725 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:02:39.028188 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:02:39.029169 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_concurrency.lockutils [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105578) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:02:39.029825 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:02:39.030534 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:02:39.030968 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:02:39.031342 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO dbcounter [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Registered counter for database nova_cell0 Oct 05 14:02:39.032930 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:02:39.035065 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO dbcounter [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Registered counter for database nova_cell0 Oct 05 14:02:39.035365 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO dbcounter [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Registered counter for database nova_cell1 Oct 05 14:02:39.039143 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO dbcounter [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Registered counter for database nova_cell1 Oct 05 14:02:39.044894 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_db.sqlalchemy.engines [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105578) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:39.045724 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_db.sqlalchemy.engines [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105578) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:39.046642 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_db.sqlalchemy.engines [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105579) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:39.050216 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_db.sqlalchemy.engines [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105579) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:39.052263 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writer thread running {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:39.053905 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writer thread running {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:39.054240 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writer thread running {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:39.188784 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writer thread running {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:39.193451 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO dbcounter [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Registered counter for database nova_cell0 Oct 05 14:02:39.197917 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO dbcounter [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Registered counter for database nova_cell0 Oct 05 14:02:39.203024 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG oslo_db.sqlalchemy.engines [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105578) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:39.205305 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_db.sqlalchemy.engines [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105579) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:02:39.208785 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writer thread running {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:39.210708 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writer thread running {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:02:39.549964 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: WARNING keystonemiddleware.auth_token [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:02:39.551234 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: WARNING keystonemiddleware.auth_token [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:02:39.557415 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: WARNING keystonemiddleware.auth_token [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:02:39.558641 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: WARNING keystonemiddleware.auth_token [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:02:39.897575 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: WARNING keystonemiddleware.auth_token [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:02:39.899099 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: WARNING keystonemiddleware.auth_token [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:02:39.903515 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: WARNING keystonemiddleware.auth_token [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:02:39.905128 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: WARNING keystonemiddleware.auth_token [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:02:39.909489 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563a0b98e9c0 pid: 105578 (default app) Oct 05 14:02:39.910433 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563a0b98e9c0 pid: 105579 (default app) Oct 05 14:02:39.911585 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:02:39.912760 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-20e7ef36-1ccc-4ab9-bd0d-0f060067abcc None None] 10.222.0.23 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001707 Oct 05 14:02:39.913180 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 788 bytes} [Wed Oct 5 14:02:39 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:01.597884 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:01.599151 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002119 Oct 05 14:04:01.599534 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:01 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:01.611370 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_api:SELECT=3 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:01.612493 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_cell1:SELECT=3 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:01.613396 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_cell0:SELECT=3 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:01.615658 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:02.528870 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-5c631c91-6de3-4b2d-b74f-11c465e1c195 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:02.562894 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-5c631c91-6de3-4b2d-b74f-11c465e1c195 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.956732 Oct 05 14:04:02.564517 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 2/3] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 5 14:04:01 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 958 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:04:05.183080 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:05.184061 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-a05c6708-6dd1-4a29-b3ed-4506048e1144 None None] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 Oct 05 14:04:05.184695 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 2/4] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:05.625836 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-9513121a-675a-4308-b88c-dd159034e247 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "os-flavor-access:is_public": true, "ram": 256, "swap": 0, "rxtx_factor": 1.0, "disk": 1, "id": "c1", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:05.673453 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-9513121a-675a-4308-b88c-dd159034e247 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.482718 Oct 05 14:04:05.674266 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 3/5] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:05 2022] POST /compute/v2.1/flavors => generated 407 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:05.688929 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_api:SELECT=3 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:05.689863 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_cell0:SELECT=3 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:05.691891 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_cell1:SELECT=3 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:05.692681 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:05.894795 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-291d257c-5aed-4e95-bea4-49d94c8728d1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:05.962318 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-291d257c-5aed-4e95-bea4-49d94c8728d1 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.279561 Oct 05 14:04:05.962911 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 3/6] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 5 14:04:05 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:04:08.532171 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-9513121a-675a-4308-b88c-dd159034e247 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:08.533357 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-9513121a-675a-4308-b88c-dd159034e247 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001649 Oct 05 14:04:08.534029 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 4/7] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:09.196572 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-3de7876f-07de-4a4c-8eba-d7bbff2b9364 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "vcpus": 1, "name": "ds512M", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 512, "disk": 5, "id": "d1"}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:09.237465 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-3de7876f-07de-4a4c-8eba-d7bbff2b9364 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.697045 Oct 05 14:04:09.238049 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 4/8] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:08 2022] POST /compute/v2.1/flavors => generated 404 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 05 14:04:09.250906 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-d3fb9311-03bf-4d39-ada0-b40d612393e0 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:09.290499 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-d3fb9311-03bf-4d39-ada0-b40d612393e0 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.043543 Oct 05 14:04:09.291097 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 5/9] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 5 14:04:09 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:04:11.904192 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-3de7876f-07de-4a4c-8eba-d7bbff2b9364 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:11.905248 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-3de7876f-07de-4a4c-8eba-d7bbff2b9364 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001551 Oct 05 14:04:11.905663 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:12.340054 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-d1582d4d-6d75-42f6-97a2-e739649ae6e1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 10, "id": "d2", "vcpus": 1, "swap": 0, "ram": 1024, "rxtx_factor": 1.0, "name": "ds1G", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:12.362677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-d1582d4d-6d75-42f6-97a2-e739649ae6e1 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.449978 Oct 05 14:04:12.363499 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 6/11] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:11 2022] POST /compute/v2.1/flavors => generated 404 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:12.375484 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-7546ef3f-6001-4ac4-916f-8fa227edb27f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:12.403454 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-7546ef3f-6001-4ac4-916f-8fa227edb27f admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031685 Oct 05 14:04:12.404056 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 6/12] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 5 14:04:12 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:04:15.002115 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-d1582d4d-6d75-42f6-97a2-e739649ae6e1 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:15.003361 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-d1582d4d-6d75-42f6-97a2-e739649ae6e1 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001782 Oct 05 14:04:15.008974 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 7/13] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:15 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:15.409354 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-40869732-6a35-4c67-a32f-bb725fedc964 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "swap": 0, "ram": 2048, "rxtx_factor": 1.0, "disk": 10, "os-flavor-access:is_public": true, "name": "ds2G", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:15.429218 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-40869732-6a35-4c67-a32f-bb725fedc964 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.417991 Oct 05 14:04:15.429979 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 7/14] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:15 2022] POST /compute/v2.1/flavors => generated 404 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:15.442034 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-e6691b87-81d7-4fa2-aaec-97165031d467 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:15.471121 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-e6691b87-81d7-4fa2-aaec-97165031d467 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032744 Oct 05 14:04:15.472035 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 8/15] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 5 14:04:15 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:04:18.200266 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-40869732-6a35-4c67-a32f-bb725fedc964 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:18.203064 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-40869732-6a35-4c67-a32f-bb725fedc964 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001990 Oct 05 14:04:18.203064 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 8/16] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:18 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:18.643115 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-3c85b267-3453-4b5b-b81f-e18195951ecf admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "swap": 0, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 4, "id": "d4", "ram": 4096, "name": "ds4G"}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:18.664735 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-3c85b267-3453-4b5b-b81f-e18195951ecf admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.454270 Oct 05 14:04:18.665308 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 9/17] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:18 2022] POST /compute/v2.1/flavors => generated 404 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:18.679021 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-d537140d-0897-4290-9228-bd2db62040b1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:18.707321 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-d537140d-0897-4290-9228-bd2db62040b1 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032264 Oct 05 14:04:18.707891 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 9/18] 10.222.0.23 () {64 vars in 1336 bytes} [Wed Oct 5 14:04:18 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:04:21.444644 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-3c85b267-3453-4b5b-b81f-e18195951ecf admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:21.445771 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-3c85b267-3453-4b5b-b81f-e18195951ecf admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001596 Oct 05 14:04:21.446165 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 10/19] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:21 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:21.862955 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-f9463775-f81a-441a-bce4-e005bcb2ae69 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:21.887809 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-f9463775-f81a-441a-bce4-e005bcb2ae69 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.434199 Oct 05 14:04:21.888510 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 10/20] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 5 14:04:21 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:04:24.861784 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-3c85b267-3453-4b5b-b81f-e18195951ecf admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:24.862844 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-3c85b267-3453-4b5b-b81f-e18195951ecf admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001546 Oct 05 14:04:24.863270 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 11/21] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:24 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:25.315706 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-5f50d73e-daa8-4aa3-a314-d49aabd56837 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.tiny", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "ram": 512, "swap": 0, "disk": 1, "os-flavor-access:is_public": true, "vcpus": 1, "id": "1"}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:25.337748 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-5f50d73e-daa8-4aa3-a314-d49aabd56837 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.468322 Oct 05 14:04:25.338282 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 11/22] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:24 2022] POST /compute/v2.1/flavors => generated 402 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:25.350367 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-677abf2d-1d06-45cf-9353-8966ed7ace15 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:25.378720 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-677abf2d-1d06-45cf-9353-8966ed7ace15 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031982 Oct 05 14:04:25.379347 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 12/23] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 5 14:04:25 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:04:27.939564 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-5f50d73e-daa8-4aa3-a314-d49aabd56837 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:27.940604 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-5f50d73e-daa8-4aa3-a314-d49aabd56837 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001501 Oct 05 14:04:27.940990 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:28.348304 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-dfa8912a-37e3-46fb-80e8-5e2687f637a1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "vcpus": 1, "id": "2", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "name": "m1.small", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 20}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:28.368850 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-dfa8912a-37e3-46fb-80e8-5e2687f637a1 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.421475 Oct 05 14:04:28.369404 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 13/25] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:27 2022] POST /compute/v2.1/flavors => generated 405 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:28.381492 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-27486cf9-91b3-4944-b084-24f0ddaa964f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:28.406897 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-27486cf9-91b3-4944-b084-24f0ddaa964f admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029074 Oct 05 14:04:28.407466 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 13/26] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 5 14:04:28 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:04:30.992126 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-dfa8912a-37e3-46fb-80e8-5e2687f637a1 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:30.993136 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-dfa8912a-37e3-46fb-80e8-5e2687f637a1 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001484 Oct 05 14:04:30.993543 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 14/27] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:31.384903 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-c51293bf-9e9c-4ee1-92fd-16119758b5db admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "vcpus": 2, "name": "m1.medium", "id": "3", "disk": 40, "ram": 4096, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:31.412710 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-c51293bf-9e9c-4ee1-92fd-16119758b5db admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.412899 Oct 05 14:04:31.426051 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-045c452d-f63e-42b1-bf4d-5ba9f37aeaf1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:31.431280 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 14/28] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:30 2022] POST /compute/v2.1/flavors => generated 406 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:31.456778 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-045c452d-f63e-42b1-bf4d-5ba9f37aeaf1 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034634 Oct 05 14:04:31.457362 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 15/29] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 5 14:04:31 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:04:34.138341 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-c51293bf-9e9c-4ee1-92fd-16119758b5db admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:34.140168 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-c51293bf-9e9c-4ee1-92fd-16119758b5db admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002395 Oct 05 14:04:34.140752 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 15/30] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:34.586146 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-a0d64397-88e6-4660-a684-2250f6596a5d admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "ram": 8192, "id": "4", "rxtx_factor": 1.0, "vcpus": 4, "os-flavor-access:is_public": true, "disk": 80, "swap": 0}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:34.593174 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:34.610831 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-a0d64397-88e6-4660-a684-2250f6596a5d admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.463814 Oct 05 14:04:34.611391 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 16/31] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:34 2022] POST /compute/v2.1/flavors => generated 405 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:34.623633 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-b5377ba8-be2e-40f2-ab0e-e07cefccd5f9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:34.648824 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-b5377ba8-be2e-40f2-ab0e-e07cefccd5f9 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028960 Oct 05 14:04:34.650104 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 16/32] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 5 14:04:34 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:04:37.237148 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-a0d64397-88e6-4660-a684-2250f6596a5d admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:37.238200 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-a0d64397-88e6-4660-a684-2250f6596a5d admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001515 Oct 05 14:04:37.238597 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 17/33] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:37.671146 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-3537aeb6-3aac-422b-b2bb-d8fe7e3cae1c admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "id": "5", "ram": 16384, "rxtx_factor": 1.0, "swap": 0, "disk": 160, "os-flavor-access:is_public": true, "name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:37.678336 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:37.697245 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-3537aeb6-3aac-422b-b2bb-d8fe7e3cae1c admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.452719 Oct 05 14:04:37.698479 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 17/34] 10.222.0.23 () {64 vars in 1283 bytes} [Wed Oct 5 14:04:37 2022] POST /compute/v2.1/flavors => generated 408 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:04:37.710618 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-e39ca1d7-0c5f-45f7-9b54-b6048801aa47 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:04:37.739490 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-e39ca1d7-0c5f-45f7-9b54-b6048801aa47 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032894 Oct 05 14:04:37.740081 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 18/35] 10.222.0.23 () {64 vars in 1333 bytes} [Wed Oct 5 14:04:37 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:04:57.190998 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-3537aeb6-3aac-422b-b2bb-d8fe7e3cae1c admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:57.192008 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-3537aeb6-3aac-422b-b2bb-d8fe7e3cae1c admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001477 Oct 05 14:04:57.192373 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 18/36] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:04:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:04:57.207016 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:57.621747 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-53371166-5957-4185-9ace-f26235961054 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:57.640101 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-53371166-5957-4185-9ace-f26235961054 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.441003 Oct 05 14:04:57.640608 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 19/37] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 5 14:04:57 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:04:58.956791 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-3537aeb6-3aac-422b-b2bb-d8fe7e3cae1c admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:58.957829 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-3537aeb6-3aac-422b-b2bb-d8fe7e3cae1c admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 Oct 05 14:04:58.958226 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 19/38] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:04:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:04:59.523308 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-453f4327-fa0b-4c09-9038-792010826b7e admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:59.530861 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.wsgi [None req-453f4327-fa0b-4c09-9038-792010826b7e admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-453f4327-fa0b-4c09-9038-792010826b7e admin admin] Request method failure captured: Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Accept: application/json Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Accept-Encoding: gzip, deflate Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Connection: keep-alive Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Content-Length: 0 Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Host: 10.222.0.23 Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Openstack-Api-Version: compute 2.92 Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Openstack-System-Scope: None Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: User-Agent: python-novaclient Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Auth-Token: gAAAAABjPY8K-g5lH04U4J72h9OWJYuZWuOG4CmvXoDSUu7aA15Pdgtvy-czT_GIkQgGTA7k4NQDFa6o_4mW_MItUS1BO-pSqm1tIIAsx3mw3kk5MSyzBFsU5ns0gWTmiThR9LHGQ-aMh_E-XBrN8GWXQ2ooo1emT2MiDed-hnAC6KSLtflMDKo Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Domain-Id: None Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Domain-Name: None Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Identity-Status: Confirmed Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Is-Admin-Project: True Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Openstack-Nova-Api-Version: 2.92 Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Domain-Id: default Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Domain-Name: Default Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Name: admin Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Role: admin,reader,member Oct 05 14:04:59.534143 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Roles: admin,reader,member Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_8abb8ea74a4d492daea0a4d1c426e1f2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant: admin Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant-Name: admin Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User: admin Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Domain-Id: default Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Domain-Name: Default Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Id: e92af64dbb734bd49c69c6f84550ccd3 Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Name: admin Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: method: > Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: exception: Flavor 42 could not be found. Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: action_args: {'id': '42'} Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:04:59.535677 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:04:59.536809 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.538173 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-453f4327-fa0b-4c09-9038-792010826b7e admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105578) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:04:59.539452 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-453f4327-fa0b-4c09-9038-792010826b7e admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.426407 Oct 05 14:04:59.539452 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 20/39] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 5 14:04:59 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 428 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:59.545596 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-89ec071b-ed88-47e5-a95e-d7f7a876e4c2 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:59.567908 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-89ec071b-ed88-47e5-a95e-d7f7a876e4c2 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.025998 Oct 05 14:04:59.568469 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 20/40] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 5 14:04:59 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:04:59.579423 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-db94a1d8-baaa-481d-9f4c-1cb46db78184 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:59.601404 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-db94a1d8-baaa-481d-9f4c-1cb46db78184 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026631 Oct 05 14:04:59.601961 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 21/41] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 5 14:04:59 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:04:59.611881 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-7b60930e-3229-43ee-952d-803285e1c084 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:04:59.617191 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.wsgi [None req-7b60930e-3229-43ee-952d-803285e1c084 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-7b60930e-3229-43ee-952d-803285e1c084 admin admin] Request method failure captured: Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Accept: application/json Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Accept-Encoding: gzip, deflate Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Connection: keep-alive Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Content-Length: 0 Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Host: 10.222.0.23 Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Openstack-Api-Version: compute 2.92 Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Openstack-System-Scope: None Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: User-Agent: python-novaclient Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Auth-Token: gAAAAABjPY8K-g5lH04U4J72h9OWJYuZWuOG4CmvXoDSUu7aA15Pdgtvy-czT_GIkQgGTA7k4NQDFa6o_4mW_MItUS1BO-pSqm1tIIAsx3mw3kk5MSyzBFsU5ns0gWTmiThR9LHGQ-aMh_E-XBrN8GWXQ2ooo1emT2MiDed-hnAC6KSLtflMDKo Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Domain-Id: None Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Domain-Name: None Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Identity-Status: Confirmed Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Is-Admin-Project: True Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Openstack-Nova-Api-Version: 2.92 Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Domain-Id: default Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Domain-Name: Default Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Name: admin Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Role: admin,reader,member Oct 05 14:04:59.620516 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Roles: admin,reader,member Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_8abb8ea74a4d492daea0a4d1c426e1f2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant: admin Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant-Name: admin Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User: admin Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Domain-Id: default Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Domain-Name: Default Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Id: e92af64dbb734bd49c69c6f84550ccd3 Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Name: admin Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: method: > Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: exception: Flavor 42 could not be found. Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: action_args: {'id': '42'} Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:04:59.622022 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:04:59.623095 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:04:59.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-7b60930e-3229-43ee-952d-803285e1c084 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105579) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:04:59.625547 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-7b60930e-3229-43ee-952d-803285e1c084 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.013618 Oct 05 14:04:59.625547 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 21/42] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 5 14:04:59 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:00.863036 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-db94a1d8-baaa-481d-9f4c-1cb46db78184 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:00.865158 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-db94a1d8-baaa-481d-9f4c-1cb46db78184 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.004063 Oct 05 14:05:00.865806 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 22/43] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:05:00 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:05:01.418625 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-424e8b38-79eb-4c53-a95d-7416041d3b9a admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:01.424025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.wsgi [None req-424e8b38-79eb-4c53-a95d-7416041d3b9a admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-424e8b38-79eb-4c53-a95d-7416041d3b9a admin admin] Request method failure captured: Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Accept: application/json Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Accept-Encoding: gzip, deflate Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Connection: keep-alive Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Content-Length: 0 Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Host: 10.222.0.23 Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Openstack-Api-Version: compute 2.92 Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Openstack-System-Scope: None Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: User-Agent: python-novaclient Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Auth-Token: gAAAAABjPY8MxREyPGYhZXuP72Cnlf_IsRazTL8siK6xx9xvAZSYj-_L7_dcIRCNQCP2G3r5uVBPhmNfJXf9bPQr6rx7JYqQnO4ABNaAAk2zvO9ecd5jW6i7qZQCwybIaDPKpR1MddgYEeRMoLdlNn8ZTMVxHdvBltfrB97EVsIvjceeCPs9xXA Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Domain-Id: None Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Domain-Name: None Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Identity-Status: Confirmed Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Is-Admin-Project: True Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Openstack-Nova-Api-Version: 2.92 Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Domain-Id: default Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Domain-Name: Default Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Name: admin Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Role: admin,reader,member Oct 05 14:05:01.425552 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Roles: admin,reader,member Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_8abb8ea74a4d492daea0a4d1c426e1f2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant: admin Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant-Name: admin Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User: admin Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Domain-Id: default Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Domain-Name: Default Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Id: e92af64dbb734bd49c69c6f84550ccd3 Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Name: admin Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: method: > Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: exception: Flavor 84 could not be found. Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: action_args: {'id': '84'} Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:05:01.427025 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:01.428038 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.429105 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-424e8b38-79eb-4c53-a95d-7416041d3b9a admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105579) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:05:01.430139 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-424e8b38-79eb-4c53-a95d-7416041d3b9a admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.424948 Oct 05 14:05:01.430139 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 5 14:05:01 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 427 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:01.437254 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-d1776af9-a12d-4104-9bd9-4c30cc57db72 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:01.452916 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-d1776af9-a12d-4104-9bd9-4c30cc57db72 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019499 Oct 05 14:05:01.453442 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 23/45] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 5 14:05:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:05:01.463251 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-17f32263-a7cf-467e-af73-6de49273fcf7 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:01.478672 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-17f32263-a7cf-467e-af73-6de49273fcf7 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019052 Oct 05 14:05:01.479529 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 5 14:05:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:05:01.488711 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-c58d8178-5968-40b7-8075-6eb725de5fc8 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:01.493754 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.wsgi [None req-c58d8178-5968-40b7-8075-6eb725de5fc8 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-c58d8178-5968-40b7-8075-6eb725de5fc8 admin admin] Request method failure captured: Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Accept: application/json Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Accept-Encoding: gzip, deflate Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Connection: keep-alive Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Content-Length: 0 Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Host: 10.222.0.23 Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Openstack-Api-Version: compute 2.92 Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Openstack-System-Scope: None Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: User-Agent: python-novaclient Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Auth-Token: gAAAAABjPY8MxREyPGYhZXuP72Cnlf_IsRazTL8siK6xx9xvAZSYj-_L7_dcIRCNQCP2G3r5uVBPhmNfJXf9bPQr6rx7JYqQnO4ABNaAAk2zvO9ecd5jW6i7qZQCwybIaDPKpR1MddgYEeRMoLdlNn8ZTMVxHdvBltfrB97EVsIvjceeCPs9xXA Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Domain-Id: None Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Domain-Name: None Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Identity-Status: Confirmed Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Is-Admin-Project: True Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Openstack-Nova-Api-Version: 2.92 Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Domain-Id: default Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Domain-Name: Default Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Name: admin Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Role: admin,reader,member Oct 05 14:05:01.495255 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Roles: admin,reader,member Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_8abb8ea74a4d492daea0a4d1c426e1f2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant: admin Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant-Name: admin Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User: admin Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Domain-Id: default Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Domain-Name: Default Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Id: e92af64dbb734bd49c69c6f84550ccd3 Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Name: admin Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: method: > Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: exception: Flavor 84 could not be found. Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: action_args: {'id': '84'} Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:05:01.496863 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:01.497774 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:01.499039 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-c58d8178-5968-40b7-8075-6eb725de5fc8 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105578) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:05:01.500225 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-c58d8178-5968-40b7-8075-6eb725de5fc8 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011371 Oct 05 14:05:01.500225 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 24/47] 10.222.0.23 () {60 vars in 1190 bytes} [Wed Oct 5 14:05:01 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:02.841779 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-17f32263-a7cf-467e-af73-6de49273fcf7 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:02.842909 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-17f32263-a7cf-467e-af73-6de49273fcf7 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001580 Oct 05 14:05:02.843302 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 24/48] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:05:02 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:05:03.398383 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-4172fc9a-8818-4a37-b7e4-880065ae7758 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:03.404493 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.wsgi [None req-4172fc9a-8818-4a37-b7e4-880065ae7758 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-4172fc9a-8818-4a37-b7e4-880065ae7758 admin admin] Request method failure captured: Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Accept: application/json Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Accept-Encoding: gzip, deflate Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Connection: keep-alive Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Content-Length: 0 Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Host: 10.222.0.23 Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Openstack-Api-Version: compute 2.92 Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Openstack-System-Scope: None Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: User-Agent: python-novaclient Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Auth-Token: gAAAAABjPY8OY6lLyovGeqSs2tUT7tZt_pg5eGfAV-oUEEGIaT0UxMv48-5gw0EJ7UBgYN68bmnGwV6I-GlXZKFiDEeN5zKABgTzS5qf5jw6ZwEkXLqYKhwIvJXhhbYW9Yps8oxfd5wAEs-kUhlJt_ztjOuL0922aa5-0viWEF9GmImo5TYTTa8 Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Domain-Id: None Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Domain-Name: None Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Identity-Status: Confirmed Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Is-Admin-Project: True Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Openstack-Nova-Api-Version: 2.92 Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Domain-Id: default Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Domain-Name: Default Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Project-Name: admin Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Role: admin,reader,member Oct 05 14:05:03.406105 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Roles: admin,reader,member Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_8abb8ea74a4d492daea0a4d1c426e1f2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant: admin Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-Tenant-Name: admin Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User: admin Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Domain-Id: default Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Domain-Name: Default Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Id: e92af64dbb734bd49c69c6f84550ccd3 Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: X-User-Name: admin Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: method: > Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: exception: Flavor 451 could not be found. Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: action_args: {'id': '451'} Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:05:03.407662 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:03.408684 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.409903 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-4172fc9a-8818-4a37-b7e4-880065ae7758 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105578) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:05:03.411160 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-4172fc9a-8818-4a37-b7e4-880065ae7758 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.425127 Oct 05 14:05:03.411160 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 25/49] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 5 14:05:02 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 427 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:03.418057 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-febd2d03-b790-4271-98dd-ba3c35a27e42 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:03.432880 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-febd2d03-b790-4271-98dd-ba3c35a27e42 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018493 Oct 05 14:05:03.433396 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 25/50] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 5 14:05:03 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:05:03.443091 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:03.458036 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] 10.222.0.23 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018493 Oct 05 14:05:03.458693 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 26/51] 10.222.0.23 () {60 vars in 1210 bytes} [Wed Oct 5 14:05:03 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:05:03.468106 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-24f9ee68-2725-4f0f-886a-95fb408ad98e admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:03.473314 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.wsgi [None req-24f9ee68-2725-4f0f-886a-95fb408ad98e admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-24f9ee68-2725-4f0f-886a-95fb408ad98e admin admin] Request method failure captured: Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Accept: application/json Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Accept-Encoding: gzip, deflate Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Connection: keep-alive Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Content-Length: 0 Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Host: 10.222.0.23 Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Openstack-Api-Version: compute 2.92 Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Openstack-System-Scope: None Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: User-Agent: python-novaclient Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Auth-Token: gAAAAABjPY8OY6lLyovGeqSs2tUT7tZt_pg5eGfAV-oUEEGIaT0UxMv48-5gw0EJ7UBgYN68bmnGwV6I-GlXZKFiDEeN5zKABgTzS5qf5jw6ZwEkXLqYKhwIvJXhhbYW9Yps8oxfd5wAEs-kUhlJt_ztjOuL0922aa5-0viWEF9GmImo5TYTTa8 Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Domain-Id: None Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Domain-Name: None Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Identity-Status: Confirmed Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Is-Admin-Project: True Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Openstack-Nova-Api-Version: 2.92 Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Domain-Id: default Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Domain-Name: Default Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Project-Name: admin Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Role: admin,reader,member Oct 05 14:05:03.474836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Roles: admin,reader,member Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_8abb8ea74a4d492daea0a4d1c426e1f2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/8abb8ea74a4d492daea0a4d1c426e1f2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant: admin Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant-Id: 8abb8ea74a4d492daea0a4d1c426e1f2 Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-Tenant-Name: admin Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User: admin Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Domain-Id: default Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Domain-Name: Default Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Id: e92af64dbb734bd49c69c6f84550ccd3 Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: X-User-Name: admin Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: method: > Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: exception: Flavor 451 could not be found. Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: action_args: {'id': '451'} Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:05:03.476548 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:03.477540 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: ERROR nova.api.openstack.wsgi Oct 05 14:05:03.478836 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-24f9ee68-2725-4f0f-886a-95fb408ad98e admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105579) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:05:03.480387 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-24f9ee68-2725-4f0f-886a-95fb408ad98e admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011650 Oct 05 14:05:03.480387 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 26/52] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 5 14:05:03 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 05 14:05:04.721654 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:04.722699 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001495 Oct 05 14:05:04.723122 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:05:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:05:05.284495 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-7f0a37b2-1ac2-4827-acb9-6a0858f5ebc4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:05:05.305921 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-7f0a37b2-1ac2-4827-acb9-6a0858f5ebc4 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.446983 Oct 05 14:05:05.306469 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 27/54] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 5 14:05:04 2022] POST /compute/v2.1/flavors => generated 404 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:05:06.546298 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:06.547305 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001549 Oct 05 14:05:06.547700 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:05:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:05:07.127993 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-1a4b45d6-14f6-4c62-9386-a212f5359200 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:05:07.153156 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-1a4b45d6-14f6-4c62-9386-a212f5359200 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.458606 Oct 05 14:05:07.153715 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 28/56] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 5 14:05:06 2022] POST /compute/v2.1/flavors => generated 405 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:05:08.381982 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:08.383008 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001524 Oct 05 14:05:08.383417 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 29/57] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:05:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:05:08.942622 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-9cd63493-6bcb-485d-b5f8-35531b74d49a admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:05:08.967202 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-9cd63493-6bcb-485d-b5f8-35531b74d49a admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.445500 Oct 05 14:05:08.967769 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 29/58] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 5 14:05:08 2022] POST /compute/v2.1/flavors => generated 407 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:05:10.189543 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:10.190597 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Oct 05 14:05:10.191000 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 30/59] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:05:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:05:10.751657 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-7cfc63c3-acc2-42b1-85c6-342e98a79df1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:05:10.774002 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-7cfc63c3-acc2-42b1-85c6-342e98a79df1 admin admin] 10.222.0.23 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.447286 Oct 05 14:05:10.774547 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 30/60] 10.222.0.23 () {64 vars in 1235 bytes} [Wed Oct 5 14:05:10 2022] POST /compute/v2.1/flavors => generated 409 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 05 14:05:12.013455 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:12.014512 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001526 Oct 05 14:05:12.014945 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 31/61] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:05:12 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:05:12.578536 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-df2260a0-66b8-4ba2-b8e3-982582a7df2f admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:05:12.598841 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-df2260a0-66b8-4ba2-b8e3-982582a7df2f admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.447669 Oct 05 14:05:12.599414 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 31/62] 10.222.0.23 () {60 vars in 1202 bytes} [Wed Oct 5 14:05:12 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:06:09.703432 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:06:09.705042 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-6a88c854-5da1-443c-9b38-774c51b86a4f admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002263 Oct 05 14:06:09.705708 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 32/63] 10.222.0.23 () {54 vars in 931 bytes} [Wed Oct 5 14:06:09 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:06:09.716401 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:10.135228 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-9a033428-a976-4a74-9fdc-d44852a52bc6 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:06:10.155726 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-9a033428-a976-4a74-9fdc-d44852a52bc6 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.443888 Oct 05 14:06:10.156253 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 32/64] 10.222.0.23 () {60 vars in 1292 bytes} [Wed Oct 5 14:06:09 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:06:10.177311 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_api:SELECT=14 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:10.181344 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-471e25cc-1ed4-4507-9aee-40622bfa3306 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:06:10.189168 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-471e25cc-1ed4-4507-9aee-40622bfa3306 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013263 Oct 05 14:06:10.189784 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 33/65] 10.222.0.23 () {60 vars in 1283 bytes} [Wed Oct 5 14:06:10 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:06:10.201614 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-25e0fa9a-e05b-4b3a-bead-a4b105cc526f admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:06:10.209321 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-25e0fa9a-e05b-4b3a-bead-a4b105cc526f admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011322 Oct 05 14:06:10.209870 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 33/66] 10.222.0.23 () {60 vars in 1286 bytes} [Wed Oct 5 14:06:10 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:06:10.221506 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:06:10.228964 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011072 Oct 05 14:06:10.229484 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 34/67] 10.222.0.23 () {60 vars in 1286 bytes} [Wed Oct 5 14:06:10 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:06:10.241373 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-24cb875c-3e25-4f15-8ac4-24c502b558b5 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:06:10.248707 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-24cb875c-3e25-4f15-8ac4-24c502b558b5 admin admin] 10.222.0.23 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010836 Oct 05 14:06:10.249247 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 34/68] 10.222.0.23 () {60 vars in 1283 bytes} [Wed Oct 5 14:06:10 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:09:14.828616 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:14.829739 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 Oct 05 14:09:14.830167 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 35/69] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:09:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:09:14.968416 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_api:SELECT=6 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:15.137492 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:15.142493 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:15.142964 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:15.143390 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:15.144099 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:15.144542 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:15.144952 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:15.176108 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-aef81bdd-5f9e-4c39-8800-e57f3901b5fa admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.209741 Oct 05 14:09:15.176655 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 35/70] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 5 14:09:14 2022] GET /compute/v2.1/os-services => generated 791 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:09:16.251317 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:16.252368 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 Oct 05 14:09:16.252780 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 36/71] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:09:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:09:16.454100 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:16.457525 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:16.458017 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:16.458434 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:16.459136 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:16.459575 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:16.459982 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:16.479668 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-0065af87-1221-496d-9c1f-a45f39d7e2eb admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.089857 Oct 05 14:09:16.480221 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 36/72] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 5 14:09:16 2022] GET /compute/v2.1/os-services => generated 791 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:09:27.999375 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:28.000381 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Oct 05 14:09:28.000789 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 37/73] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:09:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:09:28.144347 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_cell0:SELECT=4 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:28.145281 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_cell1:SELECT=4 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:28.146170 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG dbcounter [-] [105579] Writing DB stats nova_api:SELECT=8 {{(pid=105579) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:28.203081 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:28.206678 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:28.207148 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:28.207565 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:28.208263 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:28.208694 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:28.209099 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:28.234286 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-ef6985c4-a2d5-4aef-97ee-2adf1ed4d099 admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.097504 Oct 05 14:09:28.234855 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 37/74] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 5 14:09:28 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:09:29.304741 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG nova.api.openstack.wsgi [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] Calling method '>' {{(pid=105578) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:29.305788 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: INFO nova.api.openstack.requestlog [None req-12376978-61a1-44e6-ba89-a67433ce4e99 admin admin] 10.222.0.23 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Oct 05 14:09:29.306181 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: [pid: 105578|app: 0|req: 38/75] 10.222.0.23 () {56 vars in 1074 bytes} [Wed Oct 5 14:09:29 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:09:29.511024 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG nova.api.openstack.wsgi [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] Calling method '>' {{(pid=105579) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:09:29.514459 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:29.514951 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:29.515373 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:29.516083 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:09:29.516519 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:09:29.516927 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: DEBUG oslo_concurrency.lockutils [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105579) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:09:29.538682 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: INFO nova.api.openstack.requestlog [None req-cdb9b2da-c6a3-4343-8aa9-2f9f2c912c56 admin admin] 10.222.0.23 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.097010 Oct 05 14:09:29.539243 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: [pid: 105579|app: 0|req: 38/76] 10.222.0.23 () {60 vars in 1193 bytes} [Wed Oct 5 14:09:29 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:10:26.377030 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: Wed Oct 5 14:10:26 2022 - graceful shutdown triggered... Oct 05 14:10:26.377583 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: Gracefully killing worker 1 (pid: 105578)... Oct 05 14:10:26.377133 dv-cinder-835878-4-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 05 14:10:26.378474 dv-cinder-835878-4-iscsi devstack@n-api.service[105579]: Gracefully killing worker 2 (pid: 105579)... Oct 05 14:10:26.505990 dv-cinder-835878-4-iscsi devstack@n-api.service[105578]: DEBUG dbcounter [-] [105578] Writing DB stats nova_api:SELECT=4 {{(pid=105578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:10:27.379125 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: worker 1 buried after 1 seconds Oct 05 14:10:27.379125 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: worker 2 buried after 1 seconds Oct 05 14:10:27.379125 dv-cinder-835878-4-iscsi devstack@n-api.service[105577]: goodbye to uWSGI. Oct 05 14:10:27.379999 dv-cinder-835878-4-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 05 14:10:27.380884 dv-cinder-835878-4-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 05 14:10:27.385192 dv-cinder-835878-4-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 05 14:10:27.408829 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 5 14:10:27 2022] *** Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: nodename: dv-cinder-835878-4-iscsi Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: machine: x86_64 Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: clock source: unix Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: pcre jit disabled Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: detected number of CPU cores: 4 Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: current working directory: / Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: detected binary path: /usr/bin/uwsgi-core Oct 05 14:10:27.410942 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: your processes number limit is 47680 Oct 05 14:10:27.411773 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: your memory page size is 4096 bytes Oct 05 14:10:27.411773 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: detected max file descriptor number: 2048 Oct 05 14:10:27.411773 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: lock engine: pthread robust mutexes Oct 05 14:10:27.411773 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: thunder lock: enabled Oct 05 14:10:27.411773 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 05 14:10:27.411773 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 05 14:10:27.451123 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: Python main interpreter initialized at 0x5607b9c0e9c0 Oct 05 14:10:27.451123 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: python threads support enabled Oct 05 14:10:27.451123 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: your server socket listen backlog is limited to 100 connections Oct 05 14:10:27.451123 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: your mercy for graceful operations on workers is 90 seconds Oct 05 14:10:27.451560 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: mapped 403077 bytes (393 KB) for 2 cores Oct 05 14:10:27.451560 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: *** Operational MODE: preforking *** Oct 05 14:10:27.451710 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: *** uWSGI is running in multiple interpreter mode *** Oct 05 14:10:27.451710 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: spawned uWSGI master process (pid: 120590) Oct 05 14:10:27.451945 dv-cinder-835878-4-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 05 14:10:27.453679 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: spawned uWSGI worker 1 (pid: 120605, cores: 1) Oct 05 14:10:27.453679 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: spawned uWSGI worker 2 (pid: 120606, cores: 1) Oct 05 14:10:27.453679 dv-cinder-835878-4-iscsi devstack@n-api.service[120590]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 05 14:10:28.103599 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 05 14:10:28.107077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 05 14:10:29.238728 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:10:29.238728 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: from cryptography.utils import int_from_bytes Oct 05 14:10:29.239425 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:10:29.239425 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: from cryptography.utils import int_from_bytes Oct 05 14:10:29.240321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:10:29.240321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: from cryptography.utils import int_from_bytes Oct 05 14:10:29.240785 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 14:10:29.240785 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: from cryptography.utils import int_from_bytes Oct 05 14:10:31.694933 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 14:10:31.695530 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 14:10:31.695619 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 14:10:31.695966 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 14:10:31.696220 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 14:10:31.696336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 14:10:31.696698 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 14:10:31.696783 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.696973 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.697083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 14:10:31.697201 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.697427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 14:10:31.697535 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.697720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 14:10:31.697890 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.698092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.698230 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.698421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.698529 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.698763 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.698902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.699110 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.699219 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.699301 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.699376 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.699603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.699717 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.699983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.700084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835878-4-iscsi {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.700335 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.700482 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.700730 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.700948 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.701141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.701336 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.701561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.701827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.701827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.702242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835878-4-iscsi {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.702468 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.702682 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.702911 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.703121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.703121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.703516 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.703761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.703761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.703761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.704337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.704337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.704762 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.704952 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.705160 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.705372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.705582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.705582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.706047 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.706252 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.706252 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.706641 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.706825 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.707001 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.707148 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.707148 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.707432 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.707584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.708450 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.708625 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835878-4-iscsi {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.708625 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.708908 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.709062 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.709062 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.709359 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.709522 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.709711 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.709861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.710031 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835878-4-iscsi {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.710183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.710348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.710513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.710669 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.710813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.710968 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.711162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.711162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.711831 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.712004 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.712157 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.712309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.712461 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.712611 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.712762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.712905 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.713054 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.713197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.713344 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.713486 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.713626 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.713886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.714045 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.714204 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.714353 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.714503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.714651 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.714801 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.714952 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.715102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.715245 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.715385 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.715532 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.715674 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.715835 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.716008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.716008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.716714 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.716966 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.717206 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.717444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.717675 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.717675 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.719135 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.719319 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.719488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.719488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.719780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.719933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.720078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.720078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.720366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.720525 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.720679 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.720832 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.720984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.721135 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.721289 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.721431 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.721578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.721772 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.722143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.722301 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.722445 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.722586 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.722733 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.722882 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.723038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.723187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.723340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.723491 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.723640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.723789 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.723941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.724203 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.724354 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.724502 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.724644 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.724797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.724949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.725097 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.725247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.725413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.725562 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.726188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.23 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.726361 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.726521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.23 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.726672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.726821 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.726972 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.727144 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.727295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.727447 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.727597 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.727749 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.727914 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.728196 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.728463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.728675 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.728832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.729490 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.729678 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.729841 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.729995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.730484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.730634 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.730788 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.730952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.731096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.731237 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.731384 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.731533 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.731682 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.731854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.732013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.732418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.732579 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.732730 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.732873 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.733025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.733180 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.733322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.733462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.733602 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.733804 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.733963 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.734115 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.734257 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.734398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.734548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.734703 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.734861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.735015 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.735179 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.735329 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.735478 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.735628 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.735772 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.735912 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.736060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.736208 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.736349 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.736499 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.736642 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.736849 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.737068 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.737226 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.737401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.737547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.737755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.737910 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.738061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.738317 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.738492 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.738787 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.738937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.739593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.739754 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.739905 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.740060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.740219 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.740361 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.740514 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.740657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.740806 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.740948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.741106 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.741248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.741389 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.741530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.745133 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.745992 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.746159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.746490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.746647 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.746957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.747256 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.747427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.747636 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.747816 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.747959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.748100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.748247 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.748507 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.748507 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750033 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750204 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750204 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750498 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750798 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750953 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.750953 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.751239 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.751387 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.751387 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.751679 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.751831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.751831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.752110 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.752259 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.752409 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.752560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.752560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.752852 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.753000 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.753000 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.753283 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.753426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.753426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.753725 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.753869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.753869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.754161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.754305 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.754460 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.754615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.754758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.754758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.754758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.754758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 14:10:31.755533 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.755685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.755835 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.755990 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756142 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756285 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.756618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.757499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.757665 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.757833 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.757989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.758143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.758292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.758444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.758694 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.758852 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.759169 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.759320 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.759471 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.759672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.760185 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.760345 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.760501 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.760501 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.760771 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.760921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.761063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.761224 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.761414 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.761414 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.761847 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.762012 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.762156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.762306 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.762455 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.762596 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.762744 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.762893 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.763052 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.763253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.763398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.763548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.763690 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.763841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.763994 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.764147 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.764289 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.764448 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.764593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.764744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.764887 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765209 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765568 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.765765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.766840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.766991 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.767133 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.767291 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.767435 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.767580 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.767730 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.767883 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.768026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.768176 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.768318 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.768459 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.768608 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.768848 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.768998 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.769236 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.769543 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.769782 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.770327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.770487 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.770641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.770793 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.770941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.771096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.771240 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.771383 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.771525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.771680 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.771842 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.771999 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.772149 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.772298 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.772442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.772596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.772754 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.772910 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.773053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.773195 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.773336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.773478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.773626 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.773824 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.773974 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.774116 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.774273 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.774421 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.774421 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.774712 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.774855 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.775003 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.775195 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.775356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.775702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.775860 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.776022 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.776171 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.776312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.776453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.776603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.776746 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.776887 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.777028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.777176 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.777325 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.777478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.777699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.777861 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.778022 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.778172 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.778331 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.778480 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.778621 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.778887 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.779117 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.779357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.779522 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.779672 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.779862 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.780033 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.780541 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.780696 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.780856 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.781000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.781150 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.781312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.781463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.781620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.781799 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.781954 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.782104 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.782247 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.782401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.782573 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.782717 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.782867 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.783011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.783153 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.783310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.783462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.783606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.783755 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.783899 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784054 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784349 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784807 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784956 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.784956 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.785251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.785430 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.785583 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.785913 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.786074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.786233 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.786383 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.786536 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.786691 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.786843 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.786986 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.787137 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.787297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.787447 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.787590 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.787738 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.787887 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.788039 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.788182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.788331 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.788485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.788627 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.788783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.788933 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.789103 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.789345 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.789567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.789744 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.790027 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.790650 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.790811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.790963 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.791108 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.791261 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.791413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.791564 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.791715 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.791857 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.792006 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.792157 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.792317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.792468 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.792610 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.792752 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.792909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.793059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.793222 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.793372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.793516 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.793728 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.793888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.794040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.794190 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.794375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.794531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.794680 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.794822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.794972 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.795123 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.795286 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.795437 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.795587 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.795736 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.796150 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.796315 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.796460 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.796612 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.796755 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.796904 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.797056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.797207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.797358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.797512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.797692 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.797851 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.798002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.798152 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.798303 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.798453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.798603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.798761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.798904 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.799045 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.799186 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.799346 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.799593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.799820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.800982 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.801143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.801298 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.801450 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.801602 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.802036 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.802194 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.802347 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.802509 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.802664 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.802806 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.802948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.803090 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.803242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.803393 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.803544 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.803689 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.803830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.803980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.804133 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.804284 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.804447 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.804590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.804590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.804860 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.805017 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.805160 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.805312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.805477 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.805628 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.805842 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.805987 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.806351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.806502 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.806663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.806813 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.806970 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.807115 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.807264 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.807406 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.807548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.807690 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.807840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.807983 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.808131 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.808281 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.808424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.808583 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.808735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.808878 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.809018 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.809161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.809308 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.809464 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.809740 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.810117 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.810274 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.810416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.810575 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.811119 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.811283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.811433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.811575 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.811880 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.812050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.812209 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.812361 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.812504 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.812654 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.812805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.812948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.813098 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.813509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.813693 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.813857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.814001 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.814158 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.814314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.814467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.814777 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.814938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815091 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815599 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815703 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815788 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815881 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.815966 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816139 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816354 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816450 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816545 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816631 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816740 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816875 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.816970 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817155 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817340 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817425 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817607 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817873 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.817959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818147 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818331 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818422 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818506 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818596 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818680 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818762 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818762 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.818922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819103 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819187 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819268 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819268 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819583 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819669 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.819780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820084 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820084 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820301 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820301 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820605 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.820783 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821458 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821458 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821735 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.821928 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822358 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822442 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822614 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822891 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.822983 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823067 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823157 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823599 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823680 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823762 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.823844 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824006 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824093 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824176 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824453 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824538 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824628 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824804 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824895 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.824992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825145 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825256 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825424 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825570 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825737 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.825933 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826017 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826116 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826375 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826465 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826568 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826661 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826746 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826828 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.826910 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827001 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827094 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827279 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827363 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827543 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827626 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827717 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827801 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827890 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.827981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828238 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828405 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828487 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828568 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828664 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828755 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828839 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.828921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829004 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829085 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829176 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829260 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829431 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829597 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829718 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829811 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.829987 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830059 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830149 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830260 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830347 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830500 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830589 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830681 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.830859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831345 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831555 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831648 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831738 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.831922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832022 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832107 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832281 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832538 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832710 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832808 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.832989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833080 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833172 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833264 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833431 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833522 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833612 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.833944 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834035 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834137 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834306 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834482 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834670 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834752 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834834 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.834923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835006 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835279 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835457 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835634 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835726 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835909 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.835982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836058 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836207 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836275 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836707 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836862 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.836933 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837000 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837067 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837142 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837212 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837294 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837371 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837573 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837722 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.837920 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.838004 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.838218 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.838312 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.838516 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.838598 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.839981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840095 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840185 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840427 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840644 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840883 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.840962 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841142 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841461 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841565 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841700 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841887 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.841978 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.842052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.842130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.842316 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.842408 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.842628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.842782 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.842987 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.843151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.843293 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.843530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.843682 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.843869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.843981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.844177 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.844276 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.844485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.844593 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.844793 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.844895 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.845093 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.845195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.845402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.845503 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.845738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.845848 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.846039 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.846140 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.846348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.846453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.846649 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.846751 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.846956 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.847050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.847258 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.847352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.847563 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.847698 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.847877 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.848008 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.848190 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.848298 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.848498 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.848604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.848820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.848913 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.849133 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.849227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.849441 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.849529 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.849784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.849893 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.850085 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.850177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.850438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.850552 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.850699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.850807 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.851081 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.851173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.851530 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.851633 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.851716 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.851799 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.851971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.852063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.852277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.852377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.852599 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.852731 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.852900 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.853014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.853197 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.853299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.853497 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.853608 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.853969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.854076 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.854269 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.854370 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.854585 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.854686 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.854932 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.855033 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.855576 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.855679 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.855883 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.855976 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.856196 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.856316 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.856523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.856623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.856861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.856965 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.857172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.857272 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.857479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.857566 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.857833 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.857918 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.858139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.858232 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.858462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.858578 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.858770 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.858927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.859096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.859246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.859416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.859553 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.859717 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.859974 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.860075 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.860288 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.860395 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.860604 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.860723 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.860927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.861028 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.861233 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.861325 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.861590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.861778 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.861893 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.861985 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.862208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.862303 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.862519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.862612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.862843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.862937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.863168 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.863272 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.863476 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.863577 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.863776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.863882 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.864086 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.864187 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.864390 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.864490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.864693 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.864868 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.865018 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.865112 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.865318 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.865418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.865647 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.865759 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.865957 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.866084 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.866263 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.866396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.866562 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.866721 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.866862 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.867050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.867179 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.867360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.867462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.867668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.867768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.867982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.868075 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.868288 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.868395 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.868594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.868695 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.868919 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.869020 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.869225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.869343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.869537 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.869681 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.869857 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.869956 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.870159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.870265 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.870459 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.870551 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.870765 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.870867 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.871086 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.871187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.871388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.871480 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.871713 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.871817 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.872038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.872153 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.872323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.872444 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.872700 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.872800 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.873008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.873184 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.873331 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.873511 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.873657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.873849 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.873956 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.874160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.874262 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.874484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.874577 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.874784 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.874885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.875083 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.875176 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.875391 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.875504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.875690 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.875837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.875988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.876176 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.876289 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.876485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.876586 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.876816 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.876910 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.877144 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.877235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.877439 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.877543 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.877781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.877888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.878080 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.878171 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.878397 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.878491 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.878698 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.878789 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.878992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.879083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.879292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.879386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.879603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.879700 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.879890 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.879983 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.880195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.880296 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.880501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.880593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.880805 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.880931 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.881111 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.881216 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.881411 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.881506 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.881805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.881918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.882049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.882147 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.882358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.882461 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.882663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.882766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.882965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.883057 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.883267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.883360 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.883567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.883674 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.883876 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.883967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.884185 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.884278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.884496 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.884611 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.884814 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.885053 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.885145 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.885381 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.885490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.885712 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.885826 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.886025 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.886127 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.886335 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.886442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.886713 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.886809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.887024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.887124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.887330 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.887435 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.887636 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.887727 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.887942 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.888053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.888317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.888418 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.888635 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.888735 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.888950 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.889192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.889292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.889529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.889621 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.889870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.889967 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.890215 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.890374 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.890496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.890591 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.890813 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.890928 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.891128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.891240 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.891438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.891531 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.891747 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.891863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.892056 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.892181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.892367 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.892482 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.892678 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.892783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.892983 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.893122 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.893288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.893381 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.893596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.893816 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.893916 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.894009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.894220 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.894322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.894524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.894629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.894833 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.894927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.895141 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.895243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.895465 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.895567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.895794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.895893 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.896108 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.896202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.896413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.896538 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.896721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.896839 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.897028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.897144 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.897337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.897538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.897723 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.897874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.898001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.898203 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.898336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.898524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.898650 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.898846 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.898965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.899160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.899287 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.899473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.899597 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.899798 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.899924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.900100 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.900219 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.900399 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.900524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.900707 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.900833 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.901012 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.901156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.901351 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.901535 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.901687 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.901974 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.902119 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.902249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.902345 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903255 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903255 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903434 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903434 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.903819 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.904007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.904119 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.904212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.904401 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.904494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.904714 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.904820 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.905025 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.905116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.905325 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.905425 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.905670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.905790 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.905998 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.906095 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.906280 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.906374 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.906583 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.906689 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.906888 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.906985 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.907184 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.907417 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.907522 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.907770 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.907873 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.908073 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.908182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.908368 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.908469 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.908669 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.908775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.908966 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.909063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.909287 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.909384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.909586 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.909708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.909900 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.909993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.910202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.910308 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.910509 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.910612 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.910806 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.910922 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.911119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.911257 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.911459 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.911555 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.911720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.911846 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.912032 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.912158 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.912336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.912484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.912652 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.912774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.912960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.913062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.913287 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.913397 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.913596 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.913751 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.913928 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.914023 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.914238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.914341 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.914549 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.914651 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.914851 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.914945 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.915147 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.915238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.915454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.915561 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.915755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.915856 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.916064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.916157 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.916366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.916495 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.916667 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.916791 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.917009 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.917102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.917322 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.917418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.917715 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.917892 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.918030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.918220 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.918329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.918534 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.918635 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.918861 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.918971 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.919192 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.919321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.919517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.919618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.919817 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.919911 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.920114 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.920216 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.920441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.920569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.920746 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.920874 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.921055 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.921150 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.921370 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.921479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.921694 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.921953 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.922062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.922287 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.922390 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.922589 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.922684 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.922896 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.922990 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.923200 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.923307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.923508 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.923614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.923810 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.923911 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.924116 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.924211 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.924422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.924524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.924729 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.924824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.925037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.925139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.925351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.925443 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.925680 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.925794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.925984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.926087 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.926293 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.926401 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.926601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.926703 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.926905 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.926996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.927210 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.927310 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.927516 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.927607 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.927822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.927912 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.928129 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.928236 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.928438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.928529 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.928745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.928881 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.929042 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.929159 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.929383 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.929492 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.929732 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.929828 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.930054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.930157 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.930370 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.930464 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.930685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.930785 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.931008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.931110 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.931302 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.931404 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.931604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.931832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.931937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.932026 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.932216 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.932318 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.932540 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.932647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.932854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.932986 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.933161 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.933254 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.933485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.933699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.933793 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.933914 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.934098 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.934239 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.934400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.934561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.934704 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.934888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.935084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.935243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.935394 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.935566 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.935715 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.935876 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.935976 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.936202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.936309 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.936512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.936619 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.936840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.936941 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.937133 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.937235 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.937431 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.937531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.937758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.937896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.938057 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.938159 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.938359 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.938458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.938655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.938749 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.938952 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.939043 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.939248 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.939345 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.939549 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.939646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.939849 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.939953 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.940147 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.940242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.940446 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.940556 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.940742 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.940841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.941046 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.941179 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.941343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.941452 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.941670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.941799 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.941971 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.942085 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.942267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.942375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.942565 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.942658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.942866 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.943005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.943164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.943275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.943462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.943554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.943760 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.943862 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.944070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.944176 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.944369 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.944463 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.944668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.944760 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.945018 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.945121 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.945268 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.945396 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.945565 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.945727 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.945907 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.946000 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.946217 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.946311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.946526 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.946632 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.946831 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.946950 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.947160 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.947253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.947479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.947581 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.947774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.947872 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.948090 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.948190 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.948398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.948504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.948706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.948798 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.949013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.949114 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.949323 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.949423 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.949682 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.949785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.949988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.950098 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.950313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.950415 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.950656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.950759 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.950961 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.951062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.951247 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.951338 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.951566 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.951667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.951870 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.952028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.952192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.952501 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.952600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.952821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.952914 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.953137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.953335 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.953453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.953695 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.953845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.954024 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.954127 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.954335 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.954427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.954659 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.954753 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.954984 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.955085 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.955294 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.955388 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.955641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.955741 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.955963 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.956066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.956272 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.956366 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.956581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.956672 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.956897 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.956992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.957208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.957305 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.957445 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 14:10:31.957535 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.957866 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.958174 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.958511 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.958841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.959185 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.959495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.959839 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.960168 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.960480 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.960799 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.961102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 14:10:31.961343 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 14:10:32.018249 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO dbcounter [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Registered counter for database nova_api Oct 05 14:10:32.021905 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO dbcounter [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Registered counter for database nova_api Oct 05 14:10:32.040044 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_db.sqlalchemy.engines [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120606) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.042919 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_db.sqlalchemy.engines [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.049061 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writer thread running {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.049917 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writer thread running {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.109490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.context [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),003460f2-e2d0-447e-9352-d4e433e26efc(cell1) {{(pid=120605) load_cells /opt/stack/nova/nova/context.py:464}} Oct 05 14:10:32.113670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:10:32.114068 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.context [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),003460f2-e2d0-447e-9352-d4e433e26efc(cell1) {{(pid=120606) load_cells /opt/stack/nova/nova/context.py:464}} Oct 05 14:10:32.114243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:10:32.115690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:10:32.116415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:10:32.116844 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:10:32.118193 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:10:32.118660 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:10:32.118895 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:10:32.120210 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:10:32.120857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO dbcounter [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Registered counter for database nova_cell0 Oct 05 14:10:32.121010 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:10:32.121365 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:10:32.123568 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:10:32.124978 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO dbcounter [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Registered counter for database nova_cell1 Oct 05 14:10:32.125594 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO dbcounter [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Registered counter for database nova_cell0 Oct 05 14:10:32.129662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO dbcounter [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Registered counter for database nova_cell1 Oct 05 14:10:32.133657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_db.sqlalchemy.engines [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.134181 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_db.sqlalchemy.engines [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.137079 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_db.sqlalchemy.engines [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120606) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.138859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_db.sqlalchemy.engines [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120606) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.140118 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writer thread running {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.142581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writer thread running {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.144467 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writer thread running {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.147055 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writer thread running {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.285860 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO dbcounter [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Registered counter for database nova_cell0 Oct 05 14:10:32.290368 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO dbcounter [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Registered counter for database nova_cell0 Oct 05 14:10:32.293060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_db.sqlalchemy.engines [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.298206 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_db.sqlalchemy.engines [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120606) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 14:10:32.298386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writer thread running {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.303491 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writer thread running {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 14:10:32.622706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:10:32.623026 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: WARNING keystonemiddleware.auth_token [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:10:32.630034 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:10:32.630174 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: WARNING keystonemiddleware.auth_token [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:10:32.979750 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: WARNING keystonemiddleware.auth_token [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:10:32.981529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 05 14:10:32.986325 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: WARNING keystonemiddleware.auth_token [None req-5077a8e0-9772-4fb4-9241-f8d3843f30ed None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:10:32.987680 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-17aca6b1-10dc-4e4f-a9be-a61345a5d1da None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 05 14:10:32.991634 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5607b9c0e9c0 pid: 120606 (default app) Oct 05 14:10:32.993027 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5607b9c0e9c0 pid: 120605 (default app) Oct 05 14:11:06.792253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:06.794514 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:06.795640 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:06.796675 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:07.450289 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-23341239-d78f-4ec5-ac71-f0a84d912770 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Action: 'create', calling method: .version_select at 0x7f77ca7e24c0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-2033987932", "metadata": {"Type": "work"}}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:08.630550 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=3 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.631754 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=3 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.632779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=3 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.633997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=3 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.783575 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-23341239-d78f-4ec5-ac71-f0a84d912770 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 1.999807 Oct 05 14:11:08.784890 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 1/1] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 5 14:11:06 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 2004 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 05 14:11:08.805850 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5000674b-849e-4e96-92d6-12c9d4d64daa tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Calling method '.version_select at 0x7f77ca6b7430>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:08.881264 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-5000674b-849e-4e96-92d6-12c9d4d64daa tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279" status: 200 len: 353 microversion: 2.1 time: 0.079850 Oct 05 14:11:08.891557 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 2/2] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:08 2022] GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279 => generated 353 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:11:09.896249 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a73cd1b1-18c9-4786-b4fe-1fbcc0ccd19f tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Calling method '.version_select at 0x7f77ca6b7af0>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:10.024191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a73cd1b1-18c9-4786-b4fe-1fbcc0ccd19f tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279" status: 200 len: 354 microversion: 2.1 time: 0.131591 Oct 05 14:11:10.025371 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 3/3] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:09 2022] GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279 => generated 354 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:10.037827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-7ee9f38a-f81f-4b02-8e19-4e5031e0daa3 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Calling method '.version_select at 0x7f77ca64e1f0>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:10.176197 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-7ee9f38a-f81f-4b02-8e19-4e5031e0daa3 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279" status: 200 len: 354 microversion: 2.1 time: 0.141906 Oct 05 14:11:10.177324 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 4/4] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:10 2022] GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279 => generated 354 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:10.197811 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-101f5d3f-c184-4b80-a433-46a9bd2fa1cb tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Calling method '.version_select at 0x7f77ca6b7c10>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:10.209546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0676eaad-f2c7-49e8-b146-2a0d0ec90d82 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Action: 'create', calling method: .version_select at 0x7f77ca7dc4c0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1896657695", "metadata": {"Type": "work"}}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0676eaad-f2c7-49e8-b146-2a0d0ec90d82 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Request method failure captured: Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 122 Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZB8KNr3cTkwwBmMSkHfW6rJppb5BzAN8TwSQKBilGnEZglulOu-IZR6u4JdCDAhUAFa0_k6F1Zbg913QWH901qEXNuH-_1m-hyk7ykHnB7P5LT1vkCxiTneTPmNZ0SloMtlb9aAMpOI47J9rxGi-9vCLFTOh1dGg-ABmNQPaoq9IoY Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:11:10.220532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c42f92d724d24ef5a416f30a4c972857", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 35b4afac21e1469fac021bbb6fcf879e Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1896657695", "metadata": {"Type": "work"}}} Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:11:10.221901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1896657695', 'metadata': {'Type': 'work'}}}} Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise error Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi '#$%' Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:11:10.222821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 05 14:11:10.226542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 05 14:11:10.229355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 05 14:11:10.229355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.229355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0676eaad-f2c7-49e8-b146-2a0d0ec90d82 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:11:10.229355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0676eaad-f2c7-49e8-b146-2a0d0ec90d82 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.597571 Oct 05 14:11:10.242831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 1/5] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 5 14:11:08 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1614 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 05 14:11:10.247823 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c2227fe6-e970-4e94-81ed-4b9b93ffba08 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Action: 'create', calling method: .version_select at 0x7f77ca7dcdc0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1511748323", "metadata": {"Type": "work"}}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c2227fe6-e970-4e94-81ed-4b9b93ffba08 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Request method failure captured: Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 120 Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZB8KNr3cTkwwBmMSkHfW6rJppb5BzAN8TwSQKBilGnEZglulOu-IZR6u4JdCDAhUAFa0_k6F1Zbg913QWH901qEXNuH-_1m-hyk7ykHnB7P5LT1vkCxiTneTPmNZ0SloMtlb9aAMpOI47J9rxGi-9vCLFTOh1dGg-ABmNQPaoq9IoY Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:11:10.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c42f92d724d24ef5a416f30a4c972857", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 35b4afac21e1469fac021bbb6fcf879e Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1511748323", "metadata": {"Type": "work"}}} Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:11:10.253258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1511748323', 'metadata': {'Type': 'work'}}}} Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise error Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi '0' Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:11:10.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 05 14:11:10.255640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 05 14:11:10.257166 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 05 14:11:10.257166 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.257166 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c2227fe6-e970-4e94-81ed-4b9b93ffba08 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:11:10.257166 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c2227fe6-e970-4e94-81ed-4b9b93ffba08 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.016385 Oct 05 14:11:10.265828 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 2/6] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 5 14:11:10 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 26 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 05 14:11:10.273351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0641a3a8-cf76-4549-875e-cd97f196ed25 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Action: 'create', calling method: .version_select at 0x7f77ca69d1f0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-836604366", "metadata": {"Type": "work"}}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0641a3a8-cf76-4549-875e-cd97f196ed25 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Request method failure captured: Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 118 Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZB8KNr3cTkwwBmMSkHfW6rJppb5BzAN8TwSQKBilGnEZglulOu-IZR6u4JdCDAhUAFa0_k6F1Zbg913QWH901qEXNuH-_1m-hyk7ykHnB7P5LT1vkCxiTneTPmNZ0SloMtlb9aAMpOI47J9rxGi-9vCLFTOh1dGg-ABmNQPaoq9IoY Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:11:10.288863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c42f92d724d24ef5a416f30a4c972857", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 35b4afac21e1469fac021bbb6fcf879e Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-836604366", "metadata": {"Type": "work"}}} Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:11:10.290399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-836604366', 'metadata': {'Type': 'work'}}}} Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise error Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi '' Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:11:10.291340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 05 14:11:10.292729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 05 14:11:10.294261 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 05 14:11:10.294261 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:10.298613 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0641a3a8-cf76-4549-875e-cd97f196ed25 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:11:10.299823 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0641a3a8-cf76-4549-875e-cd97f196ed25 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.032986 Oct 05 14:11:10.304448 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 3/7] 10.222.0.23 () {60 vars in 1156 bytes} [Wed Oct 5 14:11:10 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 38 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 05 14:11:10.314334 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa703717-d735-4a7d-ab64-48ceb94cf165 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Calling method '.version_select at 0x7f77ca69dca0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:10.339309 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-101f5d3f-c184-4b80-a433-46a9bd2fa1cb tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279" status: 202 len: 0 microversion: 2.1 time: 0.145149 Oct 05 14:11:10.340049 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 5/8] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:11:10 2022] DELETE /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279 => generated 0 bytes in 147 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 14:11:10.348612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e8cbb585-1409-4695-9d40-438382dc233f tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Calling method '.version_select at 0x7f77ca6b7550>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:10.438544 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e8cbb585-1409-4695-9d40-438382dc233f tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279" status: 200 len: 353 microversion: 2.1 time: 0.093606 Oct 05 14:11:10.439181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 6/9] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:10 2022] GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279 => generated 353 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:11:10.691349 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-322d5ed7-2cef-46ba-b463-c3ad8187c40e tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca64e430>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:10.852363 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-322d5ed7-2cef-46ba-b463-c3ad8187c40e tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/ce3f1c43-df75-4cb8-b687-d2698b846256" status: 200 len: 351 microversion: 2.1 time: 0.170319 Oct 05 14:11:10.853110 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 7/10] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:10 2022] GET /compute/v2.1/os-volumes/ce3f1c43-df75-4cb8-b687-d2698b846256 => generated 351 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:11:11.246386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-fa703717-d735-4a7d-ab64-48ceb94cf165 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] HTTP exception thrown: Volume tempest-invalid-239431462 could not be found. Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa703717-d735-4a7d-ab64-48ceb94cf165 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Request method failure captured: Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-239431462 HTTP/1.1 Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZB8KNr3cTkwwBmMSkHfW6rJppb5BzAN8TwSQKBilGnEZglulOu-IZR6u4JdCDAhUAFa0_k6F1Zbg913QWH901qEXNuH-_1m-hyk7ykHnB7P5LT1vkCxiTneTPmNZ0SloMtlb9aAMpOI47J9rxGi-9vCLFTOh1dGg-ABmNQPaoq9IoY Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:11:11.253416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c42f92d724d24ef5a416f30a4c972857", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 35b4afac21e1469fac021bbb6fcf879e Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Volume tempest-invalid-239431462 could not be found. Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': 'tempest-invalid-239431462'} Oct 05 14:11:11.254988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-239431462 could not be found. (HTTP 404) (Request-ID: req-86ce69cd-797d-41f9-9574-364ac61172a3) Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.255916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 05 14:11:11.257183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-239431462 could not be found. Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:11:11.258278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-239431462 could not be found. Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-239431462 could not be found. Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa703717-d735-4a7d-ab64-48ceb94cf165 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Returning 404 to user: Volume tempest-invalid-239431462 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fa703717-d735-4a7d-ab64-48ceb94cf165 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/tempest-invalid-239431462" status: 404 len: 98 microversion: 2.1 time: 0.945160 Oct 05 14:11:11.259627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 4/11] 10.222.0.23 () {58 vars in 1215 bytes} [Wed Oct 5 14:11:10 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-239431462 => generated 98 bytes in 947 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 05 14:11:11.272616 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c7c9574e-7e79-42e3-822c-c9c4efb1e103 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.010270 Oct 05 14:11:11.277121 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 8/12] 10.222.0.23 () {58 vars in 1140 bytes} [Wed Oct 5 14:11:11 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 15 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 05 14:11:11.294782 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ccfeb580-f85d-4ff6-9911-f981791c0950 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.007474 Oct 05 14:11:11.295527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 5/13] 10.222.0.23 () {58 vars in 1137 bytes} [Wed Oct 5 14:11:11 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 12 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 05 14:11:11.302803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-356efb06-1c96-45d4-a96f-3a3468c4f707 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Calling method '.version_select at 0x7f77ca6b7790>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:11.454465 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-048eb88a-451d-4ebf-a5aa-a343a19759b4 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Calling method '.version_select at 0x7f77ca5d11f0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:11.533865 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-356efb06-1c96-45d4-a96f-3a3468c4f707 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] HTTP exception thrown: Volume e791935f-bceb-46e0-a90f-063b6c6477cb could not be found. Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-356efb06-1c96-45d4-a96f-3a3468c4f707 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Request method failure captured: Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: DELETE /compute/v2.1/os-volumes/e791935f-bceb-46e0-a90f-063b6c6477cb HTTP/1.1 Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZB8KNr3cTkwwBmMSkHfW6rJppb5BzAN8TwSQKBilGnEZglulOu-IZR6u4JdCDAhUAFa0_k6F1Zbg913QWH901qEXNuH-_1m-hyk7ykHnB7P5LT1vkCxiTneTPmNZ0SloMtlb9aAMpOI47J9rxGi-9vCLFTOh1dGg-ABmNQPaoq9IoY Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:11:11.538601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c42f92d724d24ef5a416f30a4c972857", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 35b4afac21e1469fac021bbb6fcf879e Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Volume e791935f-bceb-46e0-a90f-063b6c6477cb could not be found. Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': 'e791935f-bceb-46e0-a90f-063b6c6477cb'} Oct 05 14:11:11.540000 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e791935f-bceb-46e0-a90f-063b6c6477cb could not be found. (HTTP 404) (Request-ID: req-30c68598-431c-449c-bc9e-b52737742149) Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.540743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 05 14:11:11.542127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e791935f-bceb-46e0-a90f-063b6c6477cb could not be found. Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 05 14:11:11.544270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e791935f-bceb-46e0-a90f-063b6c6477cb could not be found. Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e791935f-bceb-46e0-a90f-063b6c6477cb could not be found. Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-356efb06-1c96-45d4-a96f-3a3468c4f707 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Returning 404 to user: Volume e791935f-bceb-46e0-a90f-063b6c6477cb could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-356efb06-1c96-45d4-a96f-3a3468c4f707 tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/e791935f-bceb-46e0-a90f-063b6c6477cb" status: 404 len: 109 microversion: 2.1 time: 0.241271 Oct 05 14:11:11.546413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 9/14] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:11:11 2022] DELETE /compute/v2.1/os-volumes/e791935f-bceb-46e0-a90f-063b6c6477cb => generated 109 bytes in 247 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:11:11.558984 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f8fdceaf-9903-4ae6-91e4-db8ac8026f4c tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Calling method '.version_select at 0x7f77cb010af0>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:11.628808 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-048eb88a-451d-4ebf-a5aa-a343a19759b4 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] HTTP exception thrown: Volume 2913b1e2-9e42-436f-b944-75239bfe0279 could not be found. Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-048eb88a-451d-4ebf-a5aa-a343a19759b4 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Request method failure captured: Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279 HTTP/1.1 Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZB6hD5I-rvKSoOAS8owO_wyRK_LR4Zy3EY_oZpFkM_HJHuK4XWRxOmclpX2TqT3mipNKaS8ALWWh_ZI_BYYI5ka5exdqzdxgLyqpKe6Sgk85OKWBEYVIFk-f_0kxcyzzRdp5bvo73M3kAjQ2PgzSan2g00X-h2U45lRI1AdwQTXKXU Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: fe2d7cce735d41cd8c045af9b1375ba5 Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesGetTestJSON-1139451957 Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:11:11.631309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_fe2d7cce735d41cd8c045af9b1375ba5", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/fe2d7cce735d41cd8c045af9b1375ba5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/fe2d7cce735d41cd8c045af9b1375ba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/fe2d7cce735d41cd8c045af9b1375ba5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesGetTestJSON-1139451957 Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: fe2d7cce735d41cd8c045af9b1375ba5 Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesGetTestJSON-1139451957 Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesGetTestJSON-1139451957-project Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: e6b5a3b83b904ed5831f2b4d2f22c780 Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesGetTestJSON-1139451957-project Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Volume 2913b1e2-9e42-436f-b944-75239bfe0279 could not be found. Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '2913b1e2-9e42-436f-b944-75239bfe0279'} Oct 05 14:11:11.632614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2913b1e2-9e42-436f-b944-75239bfe0279 could not be found. (HTTP 404) (Request-ID: req-7542dead-1331-4548-9591-ebc64d3fab74) Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.633415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:11:11.636283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2913b1e2-9e42-436f-b944-75239bfe0279 could not be found. Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 05 14:11:11.641246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2913b1e2-9e42-436f-b944-75239bfe0279 could not be found. Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2913b1e2-9e42-436f-b944-75239bfe0279 could not be found. Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-048eb88a-451d-4ebf-a5aa-a343a19759b4 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] Returning 404 to user: Volume 2913b1e2-9e42-436f-b944-75239bfe0279 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-048eb88a-451d-4ebf-a5aa-a343a19759b4 tempest-VolumesGetTestJSON-1139451957 tempest-VolumesGetTestJSON-1139451957-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279" status: 404 len: 109 microversion: 2.1 time: 0.185407 Oct 05 14:11:11.643912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 6/15] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:11 2022] GET /compute/v2.1/os-volumes/2913b1e2-9e42-436f-b944-75239bfe0279 => generated 109 bytes in 187 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:11:11.809338 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-f8fdceaf-9903-4ae6-91e4-db8ac8026f4c tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] HTTP exception thrown: Volume 134aaab0-20fb-479a-8799-469c3c031078 could not be found. Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f8fdceaf-9903-4ae6-91e4-db8ac8026f4c tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Request method failure captured: Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/os-volumes/134aaab0-20fb-479a-8799-469c3c031078 HTTP/1.1 Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZB8KNr3cTkwwBmMSkHfW6rJppb5BzAN8TwSQKBilGnEZglulOu-IZR6u4JdCDAhUAFa0_k6F1Zbg913QWH901qEXNuH-_1m-hyk7ykHnB7P5LT1vkCxiTneTPmNZ0SloMtlb9aAMpOI47J9rxGi-9vCLFTOh1dGg-ABmNQPaoq9IoY Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:11:11.811356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_c42f92d724d24ef5a416f30a4c972857", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/c42f92d724d24ef5a416f30a4c972857"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: c42f92d724d24ef5a416f30a4c972857 Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-VolumesNegativeTest-1071216010 Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 35b4afac21e1469fac021bbb6fcf879e Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-VolumesNegativeTest-1071216010-project Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Volume 134aaab0-20fb-479a-8799-469c3c031078 could not be found. Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '134aaab0-20fb-479a-8799-469c3c031078'} Oct 05 14:11:11.812569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 134aaab0-20fb-479a-8799-469c3c031078 could not be found. (HTTP 404) (Request-ID: req-91c6a780-b7ec-4c62-899f-cd8a98dedcb0) Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.813302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:11:11.814412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 134aaab0-20fb-479a-8799-469c3c031078 could not be found. Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 05 14:11:11.815485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 134aaab0-20fb-479a-8799-469c3c031078 could not be found. Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 134aaab0-20fb-479a-8799-469c3c031078 could not be found. Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f8fdceaf-9903-4ae6-91e4-db8ac8026f4c tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] Returning 404 to user: Volume 134aaab0-20fb-479a-8799-469c3c031078 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:11:11.818554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f8fdceaf-9903-4ae6-91e4-db8ac8026f4c tempest-VolumesNegativeTest-1071216010 tempest-VolumesNegativeTest-1071216010-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/134aaab0-20fb-479a-8799-469c3c031078" status: 404 len: 109 microversion: 2.1 time: 0.264121 Oct 05 14:11:11.822456 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 10/16] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:11 2022] GET /compute/v2.1/os-volumes/134aaab0-20fb-479a-8799-469c3c031078 => generated 109 bytes in 276 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:11:13.118161 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1d56ba5d-8696-47a0-b740-81ef0542d0d7 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca621c10>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:13.212381 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1d56ba5d-8696-47a0-b740-81ef0542d0d7 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/a12f9055-30e2-4ecd-8bf6-36cc55f28048" status: 200 len: 351 microversion: 2.1 time: 0.098472 Oct 05 14:11:13.213114 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 7/17] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:13 2022] GET /compute/v2.1/os-volumes/a12f9055-30e2-4ecd-8bf6-36cc55f28048 => generated 351 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:15.395489 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e04397d8-2d15-49f0-a3d0-4c2a632556ab tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca6383a0>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:15.525336 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e04397d8-2d15-49f0-a3d0-4c2a632556ab tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/7b351b60-8e3d-47da-8b56-4e76a42730cb" status: 200 len: 351 microversion: 2.1 time: 0.143303 Oct 05 14:11:15.526357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 11/18] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:11:15 2022] GET /compute/v2.1/os-volumes/7b351b60-8e3d-47da-8b56-4e76a42730cb => generated 351 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:15.542070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0afee637-0177-45b7-a746-f84a38b01d0f tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca5d1700>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:15.764295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0afee637-0177-45b7-a746-f84a38b01d0f tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.227374 Oct 05 14:11:15.775505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 8/19] 10.222.0.23 () {58 vars in 1134 bytes} [Wed Oct 5 14:11:15 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:15.783506 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b1fcfb87-398b-40be-8ad7-4a76ffae06b2 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca638a60>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:16.072524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b1fcfb87-398b-40be-8ad7-4a76ffae06b2 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.296056 Oct 05 14:11:16.073950 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 12/20] 10.222.0.23 () {58 vars in 1149 bytes} [Wed Oct 5 14:11:15 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:16.089486 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-21ae9404-b177-45a9-8673-d878f9b23e1e tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca5d1b80>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:16.381308 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-21ae9404-b177-45a9-8673-d878f9b23e1e tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.296857 Oct 05 14:11:16.382652 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 9/21] 10.222.0.23 () {58 vars in 1134 bytes} [Wed Oct 5 14:11:16 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:16.396487 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-de89e03e-eea0-4a4b-89f0-6b2cf95c5c68 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77cb010940>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:16.631634 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-de89e03e-eea0-4a4b-89f0-6b2cf95c5c68 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.239453 Oct 05 14:11:16.633864 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 13/22] 10.222.0.23 () {58 vars in 1167 bytes} [Wed Oct 5 14:11:16 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:16.647824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2ba191b7-1a9c-489c-8a8c-b41060cc9d87 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca621940>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:16.821426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2ba191b7-1a9c-489c-8a8c-b41060cc9d87 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.178462 Oct 05 14:11:16.830572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 10/23] 10.222.0.23 () {58 vars in 1170 bytes} [Wed Oct 5 14:11:16 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:16.836069 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-34f4c1b7-6967-48cb-a15c-6253ca394de7 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca63b790>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:17.203944 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-34f4c1b7-6967-48cb-a15c-6253ca394de7 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.372019 Oct 05 14:11:17.204744 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 14/24] 10.222.0.23 () {58 vars in 1155 bytes} [Wed Oct 5 14:11:16 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:17.222870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4f29b4a4-8e06-42a5-a11d-45f1741917d4 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca5d7f70>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:17.438513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4f29b4a4-8e06-42a5-a11d-45f1741917d4 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.223521 Oct 05 14:11:17.439102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 11/25] 10.222.0.23 () {58 vars in 1188 bytes} [Wed Oct 5 14:11:17 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:17.453372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b1e23bb5-3a45-4506-b7ee-a65f2d0bd5f0 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] Calling method '.version_select at 0x7f77ca63da60>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:17.882874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b1e23bb5-3a45-4506-b7ee-a65f2d0bd5f0 tempest-VolumesTestJSON-1400447180 tempest-VolumesTestJSON-1400447180-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.433706 Oct 05 14:11:17.897991 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 15/26] 10.222.0.23 () {58 vars in 1155 bytes} [Wed Oct 5 14:11:17 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:21.185185 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8382cc3a-e354-4583-ad77-62bd1ddf9334 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Action: 'create', calling method: .version_select at 0x7f77ca621af0>, body: {"keypair": {"name": "tempest-keypair-936939098", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD3eDysIayWTMZzj1nZjYF0cCK6ZS8Hr4REFJdbOluUN+nKgthLJuOAoM2cQOiginWE3uTAuBVTQbzuEfVAiU8FXQv33urj/YrG+VLThcpQ7+/wL7E1RBZcUqj2G9vGOZw=="}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:21.282548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-8382cc3a-e354-4583-ad77-62bd1ddf9334 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:11:21.303908 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-8382cc3a-e354-4583-ad77-62bd1ddf9334 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for user bad6116fbd5c407c9f44084fadd9531e and project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:11:21.393362 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8382cc3a-e354-4583-ad77-62bd1ddf9334 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.748427 Oct 05 14:11:21.394002 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 12/27] 10.222.0.23 () {62 vars in 1200 bytes} [Wed Oct 5 14:11:20 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 750 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 05 14:11:26.396046 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6b46d426-0bf7-425e-80d0-50b16f625afd tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Action: 'create', calling method: .version_select at 0x7f77ca63bf70>, body: {"keypair": {"name": "tempest-keypair-1649165095", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBANBu6fJsm/jWqICgPkCuRnGUaA+2KdSyy6oM2Tz33AKbsVCLcBRvpfHGig+y7QdvdOL+iDg7gCRNlQW2nWV6ChAWdsc7Ovonbd2PajVoShbr3YAsW9OVMoAFhRROJVDOQ=="}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:26.572687 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-6b46d426-0bf7-425e-80d0-50b16f625afd tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'key_pairs'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:11:26.601018 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-6b46d426-0bf7-425e-80d0-50b16f625afd tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for user 2e988e0435464c1d8aa4842fab56dfa4 and project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'key_pairs'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:11:26.641676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1638493426", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "networks": [{"uuid": "2dec276a-f9f7-496a-8e61-737556a34c9f"}]}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:26.741879 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6b46d426-0bf7-425e-80d0-50b16f625afd tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.752485 Oct 05 14:11:26.746738 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 16/28] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:11:25 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:11:26.878805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1228680426", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "58657efd-d41f-4391-a8ec-e6996bf42b73"}], "security_groups": [{"name": "tempest-securitygroup--5983715"}], "key_name": "tempest-keypair-936939098", "user_data": "}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:26.935467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.network.neutron [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] validate_networks() for [('2dec276a-f9f7-496a-8e61-737556a34c9f', None, None, None, None, None)] {{(pid=120605) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:11:27.423875 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.network.neutron [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] validate_networks() for [('58657efd-d41f-4391-a8ec-e6996bf42b73', None, None, None, None, None)] {{(pid=120606) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:11:29.126382 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Flavor limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:11:29.127098 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Image limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:11:29.127482 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Flavor pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:11:29.128125 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Image pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:11:29.128645 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:11:29.152295 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:29.153353 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:29.154127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:29.155245 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:29.156135 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:29.156644 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:29.209992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:11:29.220899 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for user bad6116fbd5c407c9f44084fadd9531e and project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:11:29.250633 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Going to run 1 instances... {{(pid=120606) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:11:29.263045 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120606) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:11:29.383705 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f4a35cec-c496-48ef-97f5-26495a0b5598 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 399 microversion: 2.20 time: 2.514585 Oct 05 14:11:29.393594 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 17/29] 10.222.0.23 () {62 vars in 1188 bytes} [Wed Oct 5 14:11:26 2022] POST /compute/v2.1/servers => generated 399 bytes in 2525 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 05 14:11:29.407762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c9947166-d174-4caf-a9c3-027780ba6837 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:29.408728 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c9947166-d174-4caf-a9c3-027780ba6837 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:29.613842 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c9947166-d174-4caf-a9c3-027780ba6837 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:29.653058 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c9947166-d174-4caf-a9c3-027780ba6837 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:29.664500 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c9947166-d174-4caf-a9c3-027780ba6837 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:29.678897 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c9947166-d174-4caf-a9c3-027780ba6837 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1281 microversion: 2.20 time: 0.283512 Oct 05 14:11:29.681993 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 18/30] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:29 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1281 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:30.175790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Flavor limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:11:30.175790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Image limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:11:30.175790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Flavor pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:11:30.175790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Image pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:11:30.175790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:11:30.208127 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:30.218614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:30.219079 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:30.220296 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:30.220981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:30.221708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:30.286460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Getting quotas for project 04395d53d99840c3b16d49e2f6d62631. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:11:30.333847 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Getting quotas for user 596b52fe5595438d8acd593c61c91f67 and project 04395d53d99840c3b16d49e2f6d62631. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:11:30.391224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Going to run 1 instances... {{(pid=120605) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:11:30.410798 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:11:30.516254 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-777874188", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b6b92f47-a2f9-4f0e-8749-335f8828b651"}], "security_groups": [{"name": "tempest-securitygroup--166092804"}], "key_name": "tempest-keypair-1649165095", "user_data": "}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:11:30.607839 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a4e14dd7-c868-4ad0-b3a9-0c5b6f3eddc5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.152761 Oct 05 14:11:30.608498 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 13/31] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:11:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 4154 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:11:30.622735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e9bd8a32-637c-45ec-87e4-1a046882b08f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:30.623486 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e9bd8a32-637c-45ec-87e4-1a046882b08f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:30.781433 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-e9bd8a32-637c-45ec-87e4-1a046882b08f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:30.794376 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.network.neutron [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] validate_networks() for [('b6b92f47-a2f9-4f0e-8749-335f8828b651', None, None, None, None, None)] {{(pid=120606) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:11:30.810868 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e9bd8a32-637c-45ec-87e4-1a046882b08f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1215 microversion: 2.1 time: 0.193823 Oct 05 14:11:30.811464 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 14/32] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:30 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1215 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:11:30.832966 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ab5a5230-306e-46fc-bb49-940be4c61762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:30.833806 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ab5a5230-306e-46fc-bb49-940be4c61762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:30.934818 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ab5a5230-306e-46fc-bb49-940be4c61762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:30.946735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ab5a5230-306e-46fc-bb49-940be4c61762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:30.952170 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ab5a5230-306e-46fc-bb49-940be4c61762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:30.954074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ab5a5230-306e-46fc-bb49-940be4c61762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1281 microversion: 2.20 time: 0.126573 Oct 05 14:11:30.957249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 15/33] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:30 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1281 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:31.835526 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-12bab38c-6966-4688-856b-7f1ef1930cec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:31.836141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-12bab38c-6966-4688-856b-7f1ef1930cec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:31.876706 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Flavor limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:11:31.876706 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Image limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:11:31.877339 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Flavor pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:11:31.877555 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Image pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:11:31.878055 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:11:31.895497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-12bab38c-6966-4688-856b-7f1ef1930cec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:31.903060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-12bab38c-6966-4688-856b-7f1ef1930cec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1215 microversion: 2.1 time: 0.071511 Oct 05 14:11:31.904445 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:31.905170 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 16/34] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:31 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1215 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:31.920498 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:31.920498 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:31.920498 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:31.920498 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:31.921020 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:31.977830 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:11:31.994960 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for user 2e988e0435464c1d8aa4842fab56dfa4 and project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:11:31.996575 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:31.998542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:32.028426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:32.030719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:32.032426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:32.035159 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:32.038232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:32.040783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:32.051030 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Going to run 1 instances... {{(pid=120606) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:11:32.064240 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120606) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:11:32.202208 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-7dc8de57-ecff-4972-8872-084605611ec4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.691290 Oct 05 14:11:32.204538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 19/35] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:11:30 2022] POST /compute/v2.1/servers => generated 401 bytes in 1697 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:11:32.219851 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2dc5f5b1-7603-430d-96c0-b572d394cef7 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:32.220660 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2dc5f5b1-7603-430d-96c0-b572d394cef7 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:32.360014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:32.370359 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2dc5f5b1-7603-430d-96c0-b572d394cef7 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:32.387574 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:32.389418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:32.397028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2f117e44-e5a6-4991-98b8-1e89dc00c65c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1333 microversion: 2.20 time: 0.411726 Oct 05 14:11:32.398633 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2dc5f5b1-7603-430d-96c0-b572d394cef7 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1241 microversion: 2.1 time: 0.170086 Oct 05 14:11:32.401861 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 20/36] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:32 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1241 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:11:32.416638 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 17/37] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:31 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1333 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:11:32.937809 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:32.940376 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:32.960711 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:32.965312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:32.968203 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:32.970756 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:32.973181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:32.975677 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:33.282398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:33.296168 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-abbf5f8f-7da7-47e4-ac18-66fe1560f9fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1267 microversion: 2.1 time: 0.370506 Oct 05 14:11:33.296866 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 21/38] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:32 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1267 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:33.427644 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:33.428273 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:33.434026 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:33.435219 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:33.437208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:33.437968 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:33.438751 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:33.439686 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:33.440346 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:33.440974 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:33.499902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:33.500422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:33.511302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:33.512131 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:33.514802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:33.515248 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:33.604855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:33.610904 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:33.615564 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fa7eeb23-9d41-4123-bc28-07120fb79292 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1237 microversion: 2.1 time: 0.195323 Oct 05 14:11:33.617623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 18/39] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:33 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1237 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:33.636593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:33.638595 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:33.641993 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-aad163b9-2aad-4958-acbd-019fd4d2cef3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1339 microversion: 2.20 time: 0.216899 Oct 05 14:11:33.642773 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 22/40] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:33 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1339 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:34.320234 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:34.321291 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:34.328608 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:34.329496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:34.330077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:34.330958 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:34.331528 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:34.331845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:34.405752 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:34.423924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-77e1d338-4a2a-473d-9441-5715b1e689bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1267 microversion: 2.1 time: 0.099937 Oct 05 14:11:34.423924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 19/41] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:34 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1267 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:34.640438 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:34.641057 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:34.647252 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:34.647717 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:34.648152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:34.648870 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:34.649317 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:34.649769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:34.673374 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:34.674177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:34.680917 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:34.681313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:34.681780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:34.686036 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:34.686719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:34.687147 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:34.716768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:34.724541 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b0a1ac12-57e3-4302-aec2-7bab00ff608c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1237 microversion: 2.1 time: 0.087830 Oct 05 14:11:34.725363 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 23/42] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:34 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1237 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:34.785262 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:34.794834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:34.796423 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:34.797375 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3fb73d8a-56fe-485b-bfba-cdfad6deafee tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1339 microversion: 2.20 time: 0.128819 Oct 05 14:11:34.797960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 20/43] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:34 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1339 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:35.441129 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:35.441980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:35.450302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:35.450946 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:35.451542 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:35.452439 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:35.453039 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:35.453597 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:35.517339 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:35.525657 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-26ae73f4-4e6d-455a-ae72-b6da16b15684 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1267 microversion: 2.1 time: 0.089471 Oct 05 14:11:35.526225 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 24/44] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:35 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1267 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:35.753314 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:35.754589 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:35.763867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:35.764394 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:35.764831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:35.765525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:35.766018 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:35.766444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:35.826951 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:35.827579 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:35.840288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:35.841096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:35.841544 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:35.842289 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:35.844687 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:35.844687 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:35.844884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:35.852393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-7fef0751-df88-426c-af1e-251707a0f724 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1237 microversion: 2.1 time: 0.105879 Oct 05 14:11:35.852972 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 21/45] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:35 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1237 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:35.908704 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:35.916804 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:35.918347 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:35.919451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a39b7001-2131-415d-a909-ce2bd4283b7a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1339 microversion: 2.20 time: 0.096973 Oct 05 14:11:35.920009 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 25/46] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:35 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1339 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:11:36.549514 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:36.550243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:36.557555 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:36.558125 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:36.558623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:36.559407 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:36.559921 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:36.560409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:36.623519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:36.631448 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-38e19276-a3dd-4aab-9227-ac107255c033 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1267 microversion: 2.1 time: 0.086581 Oct 05 14:11:36.632694 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 22/47] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:36 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1267 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:36.877058 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:36.878581 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:36.886156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:36.887352 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:36.888078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:36.888863 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:36.889390 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:36.890019 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:36.945421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:36.946332 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:36.953140 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:36.953485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:36.954320 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:36.954817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:36.955468 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:36.956082 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:36.956664 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:36.961351 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-86652475-002c-4b29-ad64-75cde8c39302 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1237 microversion: 2.1 time: 0.088146 Oct 05 14:11:36.961962 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 26/48] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:36 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1237 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:37.062211 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:37.071445 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:37.072915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:37.074060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c9097c91-b10f-4684-b0de-3fe76d468ada tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1339 microversion: 2.20 time: 0.132821 Oct 05 14:11:37.085858 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 23/49] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:36 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1339 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:37.655740 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:37.658156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:37.666896 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:37.667710 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:37.668328 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:37.669702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:37.670795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:37.671470 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:37.971617 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:37.984455 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4de62499-13f1-4ec3-a18e-d4271d1a5935 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1273 microversion: 2.1 time: 0.332543 Oct 05 14:11:37.985343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 27/50] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:37 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1273 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:37.986813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:37.987545 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:37.997267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:37.999064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:37.999930 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:38.002593 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:38.004066 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:38.005140 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:38.089253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:38.100959 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c2d6e8c6-cc48-4aad-b3a4-2b761c35a1c0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.118062 Oct 05 14:11:38.102664 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:38.102849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 24/51] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:37 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:11:38.103419 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:38.115438 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:38.116284 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:38.116805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:38.117538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:38.118030 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:38.118456 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:38.344162 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:38.359514 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:38.363154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:38.365852 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8b782cdf-67b4-4887-af43-3523241c407b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.267343 Oct 05 14:11:38.366688 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 28/52] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:38 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:11:39.014332 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:39.014902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:39.021249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:39.021789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:39.022214 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:39.022927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:39.023377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:39.023793 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:39.125393 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:39.126109 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:39.132621 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:39.133201 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:39.133909 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:39.134900 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:39.135653 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:39.136126 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:39.210066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:39.218343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a913c712-8396-484f-ae11-3205471dcdd5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.097220 Oct 05 14:11:39.218960 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 29/53] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:39 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:39.322243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:39.330712 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-504fea68-81ed-4f2b-b28b-6de6451cc8a0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.320373 Oct 05 14:11:39.331604 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 25/54] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:39 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:39.394948 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:39.395756 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:39.404081 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:39.404567 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:39.404995 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:39.405716 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:39.406168 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:39.406593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:39.537840 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:39.549181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:39.551129 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:39.553031 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-7206a15b-0b26-4cde-8787-9be1c0f91131 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.164457 Oct 05 14:11:39.554124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 30/55] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:39 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:40.244885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:40.245514 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:40.248092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:40.254192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:40.254756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:40.255284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:40.256170 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:40.256697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:40.257175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:40.322592 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:40.331026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8e47f893-b715-4a30-a002-6aa492e58deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.092082 Oct 05 14:11:40.331592 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 26/56] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:40 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:40.353424 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:40.354075 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:40.360432 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:40.360915 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:40.361366 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:40.362081 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:40.362538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:40.362959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:40.505175 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:40.514031 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-fca32015-7b66-4f62-aab1-1cd56b7064d5 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.164529 Oct 05 14:11:40.514629 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 31/57] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:40 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:40.578467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:40.579070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:40.585219 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:40.585735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:40.586164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:40.586846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:40.587291 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:40.587709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:40.717807 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:40.726207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:40.727716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:40.728852 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1984fc8a-b9c3-4cb4-a60a-2f9617def415 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.154733 Oct 05 14:11:40.729848 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 27/58] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:40 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:11:41.357846 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:41.358491 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:41.365539 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:41.366112 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:41.366596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:41.367367 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:41.367869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:41.368335 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:41.432563 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:41.440253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a69c30e8-8139-4e87-bf06-0ae67855a59a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.088631 Oct 05 14:11:41.440841 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 32/59] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:41 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:41.542211 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:41.542796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:41.552186 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:41.552760 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:41.553335 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:41.554377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:41.555023 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:41.555636 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:41.682924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:41.690855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-cd13d677-fd88-4034-8070-4c4ff7f41622 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.154965 Oct 05 14:11:41.691430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 28/60] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:41 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:41.755011 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:41.755873 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:41.771465 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:41.771950 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:41.772375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:41.773066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:41.773505 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:41.774006 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:41.910937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:41.920318 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:41.921856 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:41.923095 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f2617d6a-c0ea-4a52-ae79-1d9c0ab7486d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.171896 Oct 05 14:11:41.923970 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 33/61] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:41 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:42.463640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:42.464281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:42.466233 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=16 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:42.472209 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:42.472689 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:42.473118 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:42.473844 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:42.474300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:42.474720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:42.544620 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:42.553077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-25ffe680-6a82-4938-b4c9-33caaa83ce81 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.093369 Oct 05 14:11:42.553831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 29/62] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:42 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:42.714160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:42.714760 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:42.716631 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=4 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:42.717279 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=20 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:42.723472 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:42.723974 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:42.724400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:42.725107 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:42.725555 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:42.725996 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:42.949883 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:42.952062 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:43.002177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:43.002993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:43.003804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:43.005296 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:43.005939 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:43.006508 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:43.074328 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:43.089764 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-067b488b-b24c-4f37-8cb5-47652c016a0d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.374893 Oct 05 14:11:43.098953 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 34/63] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:42 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:11:43.212991 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:43.224037 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:43.226570 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:43.227886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-46fecbca-e1e3-47a7-893e-7186691fa53d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.282903 Oct 05 14:11:43.228517 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 30/64] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:42 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:43.576842 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:43.577463 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:43.583777 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:43.584513 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:43.585099 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:43.586017 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:43.586612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:43.597776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:43.704185 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:43.712454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-10f5e2d9-bd98-42e2-9d68-07a60e546d38 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.139647 Oct 05 14:11:43.713031 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 35/65] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:43 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:11:44.115802 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:44.116415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:44.122569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:44.123056 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:44.123486 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:44.124165 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:44.124609 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:44.125023 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:44.253473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:44.262598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:44.273019 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:44.273513 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:44.275131 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:44.275838 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:44.276290 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:44.276709 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:44.294729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:44.303148 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5a2cc060-bcae-4dd8-af3a-dcafa1491acd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.191317 Oct 05 14:11:44.303737 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 31/66] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:44 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:44.468971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:44.472545 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=100 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:44.478440 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:44.481117 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:44.482252 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-ae62f107-e8a7-424d-99ce-9b86a1686039 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.233186 Oct 05 14:11:44.482803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 36/67] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:44 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:44.736581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:44.737206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:44.743916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:44.744746 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:44.745386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:44.746488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:44.747149 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:44.747788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:44.830500 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:44.839753 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-41441b53-91b2-44ab-8bb8-769a817cd73b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.106618 Oct 05 14:11:44.840800 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 32/68] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:44 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:45.329725 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:45.330328 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:45.336620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:45.337433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:45.338102 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:45.338873 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:45.339379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:45.339930 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:45.483969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:45.492085 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-27dc31fd-5c3b-441d-b05e-3b06151bc5b2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.169096 Oct 05 14:11:45.493003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 37/69] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:45 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:45.511791 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:45.512534 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:45.518765 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:45.519258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:45.519684 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:45.520377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:45.520827 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:45.521255 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:45.525337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:45.739522 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:45.748771 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:45.750325 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:45.751568 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fd0f4904-9e58-4f4d-a9e6-8be354020c40 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.244424 Oct 05 14:11:45.752137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 33/70] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:45 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:11:45.866007 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:45.867570 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:45.876190 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:45.876681 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:45.877248 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:45.878125 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:45.878579 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:45.878994 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:45.989859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:45.997916 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-1bc8cc29-5532-4cd1-8988-e85fe7e0a296 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.137233 Oct 05 14:11:45.998480 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 38/71] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:45 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:46.517869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:46.518512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:46.524559 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:46.525037 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:46.525461 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:46.526179 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:46.526597 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:46.527110 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:46.680498 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:46.688833 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-51ae96fb-e103-437c-91fb-c2e980fdd0cd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.174823 Oct 05 14:11:46.689400 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 34/72] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:46 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:46.777246 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:46.778518 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:46.784572 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:46.785057 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:46.785486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:46.786208 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:46.786670 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:46.787073 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:46.925785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:46.953496 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:46.955198 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:46.956714 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9a83e5bf-1189-4a22-abf8-3087c5d01e4c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.183661 Oct 05 14:11:46.957414 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 39/73] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:46 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:47.023968 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:47.026047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:47.035902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:47.036413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:47.036907 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:47.037677 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:47.038087 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:47.038513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:47.109545 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:47.122459 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-cfa651a9-1367-4b8e-88a0-a5e694f2184e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.104490 Oct 05 14:11:47.123315 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 35/74] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:47 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:47.712569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:47.713871 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:47.719002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=7 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:47.724316 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:47.724867 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:47.725449 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:47.726400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:47.726972 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:47.727580 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:47.869722 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:47.877915 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e2589747-e147-499f-a8f4-b857eb8bae5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.170074 Oct 05 14:11:47.879003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 40/75] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:47 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:47.983466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:47.984108 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:47.989993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:47.990503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:47.990931 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:47.991600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:47.992043 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:47.992459 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:48.123092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:48.131392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:48.132853 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:48.133995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3bd46151-9f96-4b18-92e8-49747fc8824f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.155128 Oct 05 14:11:48.134555 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 36/76] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:47 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:48.145422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:48.146001 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:48.152227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:48.152761 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:48.153195 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:48.153950 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:48.154406 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:48.154821 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:48.217389 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:48.229716 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8e173669-5fbd-4caa-8ad6-eb2c71d11deb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.087776 Oct 05 14:11:48.230517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 41/77] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:48 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:48.902770 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:48.903716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:48.911012 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:48.911775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:48.912315 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:48.913137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:48.913713 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:48.914153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:49.066143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:49.074237 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-bf942aac-a5f7-4672-9058-9cd2659bb19a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.176586 Oct 05 14:11:49.074889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 37/78] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:48 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:49.164552 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:49.165180 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:49.171941 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:49.172431 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:49.172874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:49.173564 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:49.174044 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:49.174457 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:49.259357 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:49.260069 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:49.266486 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:49.267045 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:49.267543 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:49.268412 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:49.268859 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:49.269273 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:49.321313 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:49.331687 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:49.333285 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:49.334636 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-461369ab-70b7-4a85-bb56-d2936e4d7fd7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.173946 Oct 05 14:11:49.335415 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 42/79] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:49 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:49.351533 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:49.360577 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3b87a0bd-6b0f-4399-870d-2cc550614940 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.105191 Oct 05 14:11:49.361363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 38/80] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:49 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:50.101317 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:50.102239 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:50.113070 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:50.113566 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:50.114054 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:50.114771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:50.115218 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:50.115630 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:50.247300 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:50.256151 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-1e5d561b-1346-49d0-8c21-10d3444f3f14 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.161371 Oct 05 14:11:50.256773 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 43/81] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:50 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:50.365454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:50.365997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:50.373296 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:50.374289 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:50.375335 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:50.376458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:50.377266 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:50.378024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:50.385715 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:50.386674 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:50.393528 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:50.394122 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:50.394851 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:50.395878 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:50.396434 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:50.396917 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:50.460244 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:50.471567 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-455b3f07-d6c6-4c2b-93d4-7a4a8a10b870 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.090205 Oct 05 14:11:50.473089 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 44/82] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:50 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:50.522792 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:50.531665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:50.533146 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:50.534270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-7a21dd4c-7d37-4c3d-a237-f2b0c4193552 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.173001 Oct 05 14:11:50.534955 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 39/83] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:50 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:11:51.279360 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:51.279978 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:51.286457 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:51.286948 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:51.287374 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:51.288071 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:51.288520 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:51.288937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:51.426083 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:51.434257 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-da6a6407-2942-4961-8e06-7e78ea37d1ef tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.158847 Oct 05 14:11:51.434823 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 45/84] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:51 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:51.499307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:51.500179 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:51.502770 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:51.510195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:51.510873 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:51.511504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:51.512486 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:51.513122 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:51.513766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:51.560155 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:51.560968 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:51.567809 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:51.568343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:51.568994 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:51.569736 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:51.570226 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:51.570909 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:51.577418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:51.586446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d82de0b3-bd84-4cf7-8ef1-c8861ad1431b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.092584 Oct 05 14:11:51.601851 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 40/85] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:51 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:51.711032 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:51.723574 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:51.725835 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:51.727596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2222e88d-dcee-4526-b5dc-f0979e04802a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.172017 Oct 05 14:11:51.728418 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 46/86] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:51 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:52.457365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:52.458159 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:52.464529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:52.465019 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:52.465459 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:52.466184 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:52.466658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:52.467063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:52.601052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:52.610107 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4a3c7cc6-af9e-4708-80cb-07cc1cc2d478 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.157097 Oct 05 14:11:52.610270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:52.610662 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 41/87] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:52 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:52.610893 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:52.617049 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:52.617532 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:52.617979 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:52.618657 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:52.619084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:52.619485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:52.683353 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:52.691404 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f8ec11a0-0fe4-4804-b474-ddd392d09b8e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.084929 Oct 05 14:11:52.691959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 47/88] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:52 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:52.757706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:52.758778 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:52.768817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:52.769613 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:52.770381 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:52.771516 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:52.772271 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:52.773002 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:52.906659 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:52.915352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:52.916780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:52.918206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f36f1892-3d06-412e-b28e-3f4cb3bead13 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.166994 Oct 05 14:11:52.918781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 42/89] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:52 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:53.636949 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:53.638171 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:53.648473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:53.649279 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:53.650059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:53.651199 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:53.651952 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:53.652661 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:53.714053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:53.714682 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:53.720937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:53.721424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:53.721891 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:53.722625 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:53.723073 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:53.723497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:53.783820 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:53.784302 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:53.791655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b809e036-5aa6-40bb-9745-ac6b2777090a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.161147 Oct 05 14:11:53.792247 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 48/90] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:53 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:53.792433 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0fc08252-990e-4ace-936f-0f9582f0f7b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.082399 Oct 05 14:11:53.793015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 43/91] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:53 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:53.943299 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:53.943916 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:53.951728 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:53.952246 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:53.952709 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:53.953357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:53.953835 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:53.954258 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:54.087957 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:54.098475 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:54.100006 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:54.101138 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-db3119ff-9724-4a79-8bd8-9427ca62a5dc tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.161839 Oct 05 14:11:54.101715 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 49/92] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:53 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:54.816804 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:54.817496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:54.817914 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:54.818003 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:54.824496 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:54.824699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:54.825031 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:54.825193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:54.825501 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:54.825699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:54.826267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:54.826403 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:54.826768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:54.826885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:54.827236 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:54.827343 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:54.893177 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:54.900921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b74a03a0-b952-4811-9c27-9af58fb4b3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.088380 Oct 05 14:11:54.901572 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 50/93] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:54 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:54.959201 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:54.967127 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9b54d0f7-b371-4f59-b3c6-2d574aee407a tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.154629 Oct 05 14:11:54.967693 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 44/94] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:54 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:55.128992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:55.129595 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:55.135802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:55.136280 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:55.136700 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:55.137385 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:55.137850 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:55.138260 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:55.275575 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:55.287382 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:55.289685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:55.291307 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e0086b3f-7197-42c6-a3b4-2c7b6cdd3a0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.166592 Oct 05 14:11:55.292074 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 51/95] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:55 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:55.927191 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:55.928077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:55.938001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:55.938488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:55.938926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:55.939606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:55.940055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:55.940499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:55.992586 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:55.993111 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:55.999643 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:56.000290 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:56.000763 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:56.001470 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:56.001958 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:56.002385 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:56.005833 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:56.013767 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2fdee8e6-e7f9-4767-8fd7-a2a213f25c71 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.092288 Oct 05 14:11:56.017390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 45/96] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:55 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:56.134159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:56.141795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d8570a84-bd53-461a-b79e-31570ebf9532 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.153181 Oct 05 14:11:56.142348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 52/97] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:55 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:56.317924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:56.318443 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:56.324530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:56.325021 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:56.325452 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:56.326173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:56.326626 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:56.327044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:56.458414 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:56.466801 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:56.468253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:56.469430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b59b1b6a-8bae-455f-b517-510253f249f1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.155582 Oct 05 14:11:56.470029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 46/98] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:56 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:57.041712 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:57.042512 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:57.052620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:57.053421 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:57.054222 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:57.055315 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:57.056095 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:57.056778 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:57.078101 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=100 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:57.125860 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:57.133786 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-01b110cf-3673-4423-817a-ddb98f325573 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.098818 Oct 05 14:11:57.134517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 53/99] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:57 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:57.164092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:57.164811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:57.170951 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:57.171567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:57.171838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:57.172540 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:57.172998 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:57.173409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:57.314973 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:57.322812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e92ef0ff-a7f2-4a29-912a-50a48e14b37b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.162530 Oct 05 14:11:57.323370 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 47/100] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:57 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:57.497294 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:57.498214 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:57.504882 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:57.505422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:57.505874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:57.506598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:57.507038 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:57.507455 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:57.636765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:57.647780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:57.649959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:57.651525 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-ea53fb71-4b1e-4a74-8d1a-2b05cc528a3d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.160426 Oct 05 14:11:57.652198 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 54/101] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:57 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:58.156527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:58.157138 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:58.163619 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:58.164110 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:58.164705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:58.165218 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:58.165668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:58.166128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:58.237898 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:58.245815 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4068c124-3b02-41a5-834f-9eeb60514830 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.093189 Oct 05 14:11:58.246513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 48/102] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:58 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:58.350159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:58.350989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:58.360843 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:58.361331 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:58.361837 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:58.362566 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:58.363021 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:58.363467 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:58.501323 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:58.513220 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-594e546f-db12-4e87-a4fd-d52622b92c43 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.169805 Oct 05 14:11:58.513818 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 55/103] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:58 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:58.676625 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:58.677238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:58.683890 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:58.684381 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:58.684814 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:58.685484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:58.685977 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:58.686398 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:58.815741 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:58.819151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:58.825226 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:58.826696 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:58.827813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d2181e6e-4585-4c2c-8cc8-1f92ba4e012e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.155127 Oct 05 14:11:58.828438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 49/104] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:58 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:11:59.271643 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:59.272183 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:59.278594 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:59.279080 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:59.279514 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:59.280198 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:59.280645 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:59.281061 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:59.346151 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:59.354313 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-dce1be4f-ae9d-4d65-8da2-95a99b06b79d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.086343 Oct 05 14:11:59.354914 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 56/105] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:59 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:59.537388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:59.538030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:59.544688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:59.545164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:59.545590 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:59.546289 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:59.546729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:59.547145 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:59.680217 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:11:59.688430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d88affc0-68ba-4819-bf79-9d750cfcf3c6 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.154847 Oct 05 14:11:59.689033 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 50/106] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:11:59 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:11:59.854830 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:11:59.855580 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:11:59.862954 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:59.863433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:59.863857 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:11:59.864539 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:11:59.864988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:11:59.865403 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:00.000069 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:00.008532 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:00.010008 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:00.011493 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-488b08ff-2472-449c-bece-a91230c45c2d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.161891 Oct 05 14:12:00.012052 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 57/107] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:11:59 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:00.378020 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:00.378659 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:00.384925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:00.385416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:00.385911 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:00.386600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:00.387045 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:00.387446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:00.451514 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:00.459322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5fe447d0-7dc3-4e1c-8267-a81152b5284e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.086328 Oct 05 14:12:00.459887 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 51/108] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:00 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:00.715170 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:00.717144 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:00.727461 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:00.728196 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:00.728802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:00.730343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:00.731078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:00.731762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:00.860237 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:00.868519 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-de04347c-e202-4836-bdf1-ef58ea5d882f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.159596 Oct 05 14:12:00.869097 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 58/109] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:00 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:01.036821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:01.037380 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:01.044056 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:01.044583 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:01.045022 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:01.045762 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:01.046208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:01.046630 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:01.183921 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:01.192969 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:01.194761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:01.196035 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8aea09b2-83b4-4648-b42c-2ea8c1fa61cd tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.163201 Oct 05 14:12:01.196571 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 52/110] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:01 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:01.482288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:01.482981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:01.488988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:01.489490 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:01.489947 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:01.490618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:01.491063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:01.491479 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:01.555524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:01.563574 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a6c8f905-29bc-4c1b-ac3f-23801aec461b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.085273 Oct 05 14:12:01.564141 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 59/111] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:01 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:01.891603 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:01.892226 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:01.898541 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:01.899027 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:01.899458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:01.900138 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:01.900585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:01.901000 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:02.036913 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:02.045010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-366cf46c-9868-4fb4-8547-7255ec76ebe3 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.157378 Oct 05 14:12:02.045588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 53/112] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:01 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:02.220624 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:02.222229 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:02.228603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:02.229476 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:02.229877 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:02.230567 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:02.231013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:02.231423 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:02.360270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:02.368484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:02.369989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:02.371100 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-cb682aa9-29f5-43aa-a5ad-1a9e1546c53f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.154454 Oct 05 14:12:02.371658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 60/113] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:02 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:02.586371 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:02.586978 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:02.593134 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:02.593620 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:02.594078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:02.594765 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:02.595209 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:02.595601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:02.662955 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:02.670704 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-08705b0f-c96d-4827-a248-0d69c6d04cb5 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.088177 Oct 05 14:12:02.671262 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 54/114] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:02 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:03.070909 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:03.071664 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:03.079481 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:03.080082 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:03.080615 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:03.081462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:03.082062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:03.082585 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:03.216416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:03.224524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2b15e333-7c10-41a3-8e27-ca29cfaf3dc0 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.158810 Oct 05 14:12:03.225091 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 61/115] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:03 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:03.396669 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:03.397254 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:03.404024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:03.404511 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:03.404939 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:03.405678 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:03.406162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:03.406578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:03.543895 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:03.552406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:03.553880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:03.554977 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b1bd62a5-1f72-4e42-a46b-2b6d978919d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.162288 Oct 05 14:12:03.555567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 55/116] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:03 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:03.693922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:03.694667 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:03.700797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:03.701305 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:03.701753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:03.702437 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:03.702878 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:03.703289 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:03.766760 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:03.774638 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a7fbbb48-ff34-4cdf-ba7b-0bcae04f9319 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.084718 Oct 05 14:12:03.775191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 62/117] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:03 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:04.258667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:04.259592 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:04.269055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:04.269915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:04.270280 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:04.270964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:04.271405 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:04.271803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:04.402159 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:04.410017 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-165e69ca-3fdf-4984-88c7-c2ef211463b7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.157709 Oct 05 14:12:04.410551 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 56/118] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:04 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:04.584409 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:04.585242 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:04.595745 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:04.596547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:04.597273 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:04.598275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:04.598730 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:04.599150 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:04.742533 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:04.750984 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:04.752467 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:04.753583 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-064854dd-4df9-4ac3-bc54-bcd3cb523fb0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.176010 Oct 05 14:12:04.754281 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 63/119] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:04 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:04.796544 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:04.797150 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:04.803399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:04.803883 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:04.804309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:04.804989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:04.805432 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:04.805878 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:04.875238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:04.883232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fbdab779-9efa-4be5-9875-41d5d57a1c36 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.090640 Oct 05 14:12:04.883785 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 57/120] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:04 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:05.436707 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:05.437757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:05.445332 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:05.445880 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:05.446307 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:05.446989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:05.447431 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:05.447841 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:05.579652 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:05.589036 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-693d6458-1135-4db2-a3cf-2691b854c291 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.159306 Oct 05 14:12:05.589673 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 64/121] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:05 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:05.783672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:05.784651 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:05.795238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:05.796025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:05.797262 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:05.797911 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:05.798654 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:05.799352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:05.906604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:05.908071 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:05.914767 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:05.915568 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:05.915997 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:05.917066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:05.917492 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:05.918096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:05.930114 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:05.939656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:05.941612 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:05.942938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6dbbd174-3b2b-476c-b371-bafcaf3fe4e0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.165693 Oct 05 14:12:05.943821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 58/122] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:05 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:05.998495 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:06.006389 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-fa4fe777-6c35-4481-be18-3a8dd7a42f4e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.103735 Oct 05 14:12:06.006939 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 65/123] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:05 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:06.614601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:06.615130 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:06.621097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:06.621579 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:06.622041 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:06.622754 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:06.623199 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:06.623613 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:06.761786 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:06.773384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-763ce66e-f8b7-46d2-a233-92f813b20575 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.164199 Oct 05 14:12:06.774467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 59/124] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:06 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:06.968177 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:06.968781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:06.974933 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:06.975351 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:06.975776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:06.976458 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:06.976900 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:06.977695 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:07.036120 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:07.036824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:07.043090 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:07.043664 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:07.044105 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:07.044783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:07.045234 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:07.045659 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:07.108199 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:07.111692 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:07.117766 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:07.119191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:07.119433 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b25166ce-6cba-4e1c-81fa-e18beffd88ce tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.087403 Oct 05 14:12:07.133724 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 60/125] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:07 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:12:07.133986 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-94fb078c-43f7-4afd-b3a5-463372a2923a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.156121 Oct 05 14:12:07.134160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 66/126] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:06 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:07.796657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:07.797256 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:07.804933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:07.805610 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:07.806209 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:07.807192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:07.807794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:07.808427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:07.958040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:07.965915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ae8c84b3-26b8-48d4-b63a-5953679fdf5b tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.173140 Oct 05 14:12:07.966497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 61/127] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:07 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:08.147682 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:08.148297 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:08.154421 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:08.154904 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:08.155375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:08.155999 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:08.156439 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:08.156848 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:08.158233 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:08.158841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:08.165615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:08.166349 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:08.167007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:08.168004 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:08.168671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:08.169296 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:08.228353 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:08.238607 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-713fc20b-9394-4d23-a00c-550a06b3c11a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.098307 Oct 05 14:12:08.239227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 67/128] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:08 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:08.312488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:08.322407 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:08.323832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:08.324932 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6ec266e4-01b5-4643-9468-b7cc49c77ae6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.170707 Oct 05 14:12:08.325729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 62/129] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:08 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:12:08.994263 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:08.995372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:09.004902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:09.005508 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:09.006040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:09.006718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:09.007183 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:09.007570 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:09.150573 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:09.159027 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-27142b99-8f6f-4d2e-9150-33e612f7f569 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.171587 Oct 05 14:12:09.159566 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 68/130] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:08 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:09.265953 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:09.266465 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:09.275841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:09.276454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:09.277113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:09.278153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:09.278802 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:09.279369 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:09.348845 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:09.350199 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:09.351604 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:09.356992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:09.357465 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:09.358051 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:09.358730 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:09.359171 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:09.359573 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:09.359710 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fd461375-4b94-4bdc-ae32-17b04d8fdce2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.100180 Oct 05 14:12:09.360284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 63/131] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:09 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:09.489328 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:09.497848 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:09.499273 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:09.500379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d5705783-e7ca-4a7c-b86d-0acdd9b8a188 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.155450 Oct 05 14:12:09.500923 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 69/132] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:09 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:10.181819 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:10.182423 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:10.188833 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:10.189322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:10.189916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:10.190606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:10.191056 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:10.191474 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:10.326544 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:10.334383 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-503738a3-638e-443f-ab18-d165ad013eed tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.156379 Oct 05 14:12:10.334906 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 64/133] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:10 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:10.382178 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:10.382800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:10.388739 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:10.389221 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:10.389659 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:10.390333 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:10.390771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:10.391181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:10.395023 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=100 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:10.452282 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:10.460400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0c70d059-7144-4650-9f5d-0c86cd3e6309 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.082311 Oct 05 14:12:10.460959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 70/134] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:10 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:10.524716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:10.525309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:10.531761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:10.532270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:10.532687 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:10.533362 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:10.533837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:10.534247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:10.662255 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:10.671015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:10.672434 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:10.673526 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-13987056-7017-409f-8235-d77e29c15091 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.152772 Oct 05 14:12:10.674126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 65/135] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:10 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:11.356845 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:11.357526 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:11.363775 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:11.364256 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:11.364667 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:11.365346 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:11.365833 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:11.366253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:11.483996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:11.484576 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:11.490784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:11.491524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:11.492048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:11.492741 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:11.493216 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:11.493658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:11.497493 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:11.505911 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0a755b5f-7617-4d6c-bab6-d63e3596fb5c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.153268 Oct 05 14:12:11.506553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 71/136] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:11 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:11.513295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:11.560261 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:11.569255 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-af25cbad-9b01-46b0-bd59-93dab9e84f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.089133 Oct 05 14:12:11.570152 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 66/137] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:11 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:11.702961 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:11.703840 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:11.713992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:11.714803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:11.715530 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:11.716650 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:11.717367 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:11.718171 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:11.850750 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:11.859171 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:11.860615 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:11.861758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8b397144-9573-4b89-83a9-cf133607fe0a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.165595 Oct 05 14:12:11.862303 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 72/138] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:11 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:12.533703 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:12.534820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:12.543015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:12.543507 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:12.543938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:12.544628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:12.545075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:12.545490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:12.592973 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:12.593676 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:12.600691 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:12.601263 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:12.601785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:12.602540 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:12.603046 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:12.604225 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:12.664701 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:12.672515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-036faec3-010a-48e2-86f3-ddd277fa5275 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.083641 Oct 05 14:12:12.673074 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 73/139] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:12 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:12:12.687210 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:12.695497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0dbb8cc2-49de-4b97-b2ed-aefc48bdf91f tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.168520 Oct 05 14:12:12.696089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 67/140] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:12 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:12.891589 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:12.892448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:12.904622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:12.905104 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:12.905523 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:12.906238 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:12.906678 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:12.907086 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:13.050733 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:13.059066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:13.060484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:13.061620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9258ecb2-cb43-449a-9c17-2e713b41d451 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.177597 Oct 05 14:12:13.062223 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 74/141] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:12 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:13.699477 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:13.700590 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:13.709908 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:13.710403 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:13.710817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:13.711499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:13.711943 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:13.712354 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:13.718612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:13.719215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:13.725561 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:13.726075 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:13.726500 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:13.727187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:13.727633 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:13.728046 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:13.775029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:13.782719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-21a9c7d1-6b56-43dc-951c-3cf479ecdaec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.089628 Oct 05 14:12:13.783286 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 68/142] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:13 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:13.860248 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:13.868095 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-35def7c4-844f-4737-ac35-cfaee9d82068 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.153402 Oct 05 14:12:13.868654 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 75/143] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:13 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:14.086428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:14.087048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:14.093506 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:14.094031 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:14.094451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:14.095125 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:14.095567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:14.095973 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:14.226554 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:14.235489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:14.236914 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:14.238025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1111aea5-1ea4-42d6-b4a2-c37e0183f669 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.155683 Oct 05 14:12:14.238591 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 69/144] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:14 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:14.809870 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:14.810840 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:14.819536 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:14.820010 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:14.820432 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:14.821100 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:14.821541 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:14.821984 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:14.891113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:14.892256 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:14.892413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:14.898758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:14.899244 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:14.899667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:14.899846 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-babd3ea4-c306-468f-91ce-c09d20cb61ec tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.096408 Oct 05 14:12:14.900356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:14.900451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 76/145] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:14 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:14.900808 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:14.901228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:15.027647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:15.039859 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-56dd4dd3-fe8a-460f-919f-61bc0c8c85e2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.152377 Oct 05 14:12:15.040673 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 70/146] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:14 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:15.264079 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:15.264780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:15.270817 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:15.271357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:15.271766 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:15.272457 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:15.272897 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:15.273330 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:15.414719 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:15.423536 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:15.424986 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:15.426514 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9620bdb6-a268-463c-b827-7a440982d175 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.166289 Oct 05 14:12:15.427199 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 77/147] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:15 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:15.927524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:15.928173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:15.935169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:15.935679 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:15.936132 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:15.936873 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:15.937335 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:15.937807 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:16.001277 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:16.012800 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fa1be11d-9130-4b79-bdf4-8ad4f3b76bef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.092145 Oct 05 14:12:16.013727 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 71/148] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:15 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:16.062827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:16.063426 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:16.069816 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:16.070294 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:16.070713 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:16.071381 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:16.071825 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:16.072239 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:16.212381 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:16.220171 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b3342190-1f97-47e6-8ef2-d148a57aae06 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.161168 Oct 05 14:12:16.220751 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 78/149] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:16 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:16.455649 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:16.456743 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:16.463510 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:16.463993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:16.464418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:16.465100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:16.465545 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:16.466097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:16.607112 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:16.616522 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:16.617974 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:16.619363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-54c1f7ca-e824-4fb9-8ed7-a08e6b61c067 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.170025 Oct 05 14:12:16.620064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 72/150] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:16 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:17.042358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:17.043441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:17.053568 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:17.054416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:17.055116 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:17.056235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:17.056964 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:17.057694 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:17.127383 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:17.139534 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-20197186-fd21-4ce7-b3ab-97b2158cf671 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.103679 Oct 05 14:12:17.140113 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 79/151] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:17 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:17.248028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:17.249190 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:17.259320 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:17.259971 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:17.260497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:17.261465 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:17.262107 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:17.263252 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:17.397427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:17.405289 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1ab6dff4-21bd-4e4b-9d80-24154897a995 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.164225 Oct 05 14:12:17.405889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 73/152] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:17 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:17.648357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:17.649337 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:17.659810 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:17.660585 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:17.661317 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:17.662504 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:17.662978 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:17.663361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:17.813556 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:17.821965 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:17.823421 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:17.824527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9c494fe6-8981-49f7-b711-221b59faefca tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.182605 Oct 05 14:12:17.825056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 80/153] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:17 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:18.166540 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:18.167521 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:18.178428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:18.178888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:18.179332 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:18.180016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:18.180456 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:18.180863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:18.244174 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:18.252468 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-de73db50-04ca-4768-8af1-a6a5b1276e87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.092642 Oct 05 14:12:18.253364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 74/154] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:18 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:18.428829 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:18.429472 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:18.431557 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=82 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:18.436777 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:18.437746 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:18.438333 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:18.439186 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:18.439909 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:18.440466 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:18.579602 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:18.587327 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6ef8d7d4-7354-46d5-87fb-f33caebc2515 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.162625 Oct 05 14:12:18.588058 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 81/155] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:18 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:18.849061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:18.849693 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:18.856503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:18.856977 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:18.857399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:18.858106 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:18.858554 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:18.858960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:18.992244 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:19.000586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:19.002038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:19.003307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-18ccaf44-f5ed-4f1b-a906-5a64f987ccc8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.158071 Oct 05 14:12:19.003723 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 75/156] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:18 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:19.280032 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:19.280899 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:19.289683 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:19.290181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:19.290594 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:19.291266 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:19.291702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:19.292112 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:19.357019 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:19.365082 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-dc9009b7-f5c6-4524-9472-b4248b9799dc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.091599 Oct 05 14:12:19.365673 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 82/157] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:19 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:19.610497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:19.611242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:19.617672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:19.618188 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:19.618622 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:19.619317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:19.619768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:19.620189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:19.754260 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:19.762273 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d23ae393-7f1e-4837-b444-c2c3b4837f8e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.155605 Oct 05 14:12:19.762856 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 76/158] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:19 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:20.031377 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:20.032059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:20.040290 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:20.040769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:20.041188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:20.041901 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:20.042351 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:20.042757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:20.183248 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:20.195704 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:20.197971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:20.199622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-ea4cec10-aee5-47d1-942a-38357e67e55c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.174616 Oct 05 14:12:20.200405 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 83/159] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:20 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:20.388662 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:20.389185 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:20.395601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:20.396086 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:20.396561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:20.397238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:20.397688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:20.398134 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:20.469989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:20.478107 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6d56a14f-fdcf-43e6-a6f2-653acc2b03aa tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.093675 Oct 05 14:12:20.478996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 77/160] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:20 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:20.788831 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:20.789420 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:20.795478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:20.795971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:20.796386 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:20.797119 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:20.797512 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:20.797988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:20.927671 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:20.935439 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-7726ec95-5411-49e1-99ad-e9466a7be7ae tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.152617 Oct 05 14:12:20.936182 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 84/161] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:20 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:21.233060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:21.234036 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:21.243170 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:21.243647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:21.244067 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:21.244736 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:21.245178 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:21.245595 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:21.385217 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:21.394835 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:21.396279 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:21.397395 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-01ddb4aa-b2ad-4132-99ee-21eb81a2d27e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.171595 Oct 05 14:12:21.398178 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 78/162] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:21 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:21.502697 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:21.502697 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:21.508854 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:21.509341 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:21.509796 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:21.510523 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:21.510929 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:21.511342 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:21.574963 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:21.582884 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a8884310-0b38-467e-b7aa-c7c055ea9d2b tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.085137 Oct 05 14:12:21.583442 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 85/163] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:21 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:21.958828 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:21.959513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:21.961390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=78 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:21.966893 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:21.967367 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:21.967790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:21.968463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:21.968902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:21.969314 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:22.097273 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:22.105747 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b04a5116-000c-4d6d-8a8e-2bccdf9c2266 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.150782 Oct 05 14:12:22.106306 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 79/164] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:21 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:22.427320 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:22.428414 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:22.435119 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:22.435601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:22.436024 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:22.436752 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:22.437200 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:22.437612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:22.567464 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:22.571040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=100 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:22.576652 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:22.578119 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:22.579188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-bf1b3dae-e2bc-45e6-9201-b453223087d6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.158731 Oct 05 14:12:22.579734 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 86/165] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:22 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:22.606382 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:22.606981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:22.613439 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:22.613944 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:22.614368 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:22.615041 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:22.615481 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:22.615931 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:22.690532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:22.698543 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-870bfc8c-a321-450d-a6db-81de94400a1c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.096436 Oct 05 14:12:22.699068 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 80/166] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:22 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:23.128520 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:23.129120 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:23.135665 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:23.136179 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:23.136604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:23.137290 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:23.137746 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:23.138167 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:23.273140 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:23.281163 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f1ccd6f5-e22c-4b4a-b87d-fd7202a4b0b9 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.156468 Oct 05 14:12:23.281755 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 87/167] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:23 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:23.605305 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:23.606070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:23.612749 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:23.613344 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:23.613692 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:23.614477 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:23.614955 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:23.615393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:23.723034 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:23.723910 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:23.731531 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:23.731992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:23.732404 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:23.733089 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:23.733547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:23.734002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:23.763431 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:23.773851 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:23.776610 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:23.778353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f2dcd49a-406a-45f0-b2ca-542a0f0d3980 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.177543 Oct 05 14:12:23.779174 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 81/168] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:23 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:23.805253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:23.815750 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f9e9499f-8965-4646-9ffd-3f7a0832b3da tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.097404 Oct 05 14:12:23.816485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 88/169] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:23 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:24.307295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:24.308274 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:24.317553 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:24.318463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:24.319111 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:24.319792 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:24.320241 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:24.320654 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:24.467332 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:24.475092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3dbbbe46-8890-4ecb-b217-6aef25df0ff2 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.173878 Oct 05 14:12:24.475654 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 82/170] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:24 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:24.811411 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:24.812459 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:24.822561 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:24.823351 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:24.824056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:24.825176 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:24.825983 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:24.826677 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:24.839892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:24.840492 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:24.846380 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:24.846858 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:24.847251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:24.847920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:24.848359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:24.848766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:24.852924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:24.924749 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:24.932343 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a259e8e6-73a1-49f5-9868-06a11cf53780 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.096257 Oct 05 14:12:24.932948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 83/171] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:24 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:24.968936 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:24.977515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:24.979365 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:24.980461 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c00bd23f-fd19-4fd2-a3c8-0986d81220b6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.175765 Oct 05 14:12:24.981145 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 89/172] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:24 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:25.512242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:25.513720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:25.523456 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:25.523940 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:25.524375 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:25.525053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:25.525495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:25.525944 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:25.652078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:25.660370 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a82d63af-091c-402f-a968-f42c4c0a2434 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.154815 Oct 05 14:12:25.660909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 84/173] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:25 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:25.956179 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:25.956783 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:25.963216 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:25.963700 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:25.964123 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:25.964813 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:25.965259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:25.965734 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:26.006975 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:26.006975 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:26.012627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:26.013408 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:26.013836 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:26.014526 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:26.014970 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:26.015374 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:26.030285 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:26.044763 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-43cda18d-6bcc-434f-8385-3ebe8510ed04 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.092161 Oct 05 14:12:26.054511 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 90/174] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:25 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:26.169456 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:26.177606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:26.179155 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:26.180264 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-479805b1-2659-442b-a5f0-9031b3d7136e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.178786 Oct 05 14:12:26.180831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 85/175] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:26 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:26.686753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:26.687799 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:26.696627 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:26.697106 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:26.697533 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:26.698249 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:26.698701 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:26.699114 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:26.839974 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:26.851757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d393ff92-3308-45ea-8306-c4d8c276ea21 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.171622 Oct 05 14:12:26.852299 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 91/176] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:26 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:27.068508 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:27.069067 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:27.075493 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:27.075991 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:27.076413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:27.077090 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:27.077534 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:27.077996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:27.148516 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:27.156684 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-79e99e4e-e62b-4fb3-bcce-f8de11c5c6cc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.091974 Oct 05 14:12:27.157241 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 86/177] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:27 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:27.204898 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:27.205526 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:27.211766 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:27.212252 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:27.212684 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:27.213374 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:27.213837 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:27.214253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:27.341190 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:27.349926 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:27.351485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:27.352590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-df5d8804-500c-46a0-a596-023627ac202e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.151931 Oct 05 14:12:27.353155 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 92/178] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:27 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:27.876013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:27.876650 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:27.883487 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:27.883984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:27.884410 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:27.885097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:27.885549 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:27.886001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:28.016675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:28.024891 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-88d3c2f5-2830-496d-9504-665128efe451 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.152886 Oct 05 14:12:28.025396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 87/179] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:27 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:28.180523 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:28.181140 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:28.187352 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:28.187837 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:28.188266 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:28.188944 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:28.189386 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:28.189845 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:28.252101 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:28.260561 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-95b3ad18-ada1-4ab7-81e3-4ef428b9779e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.083903 Oct 05 14:12:28.261103 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 93/180] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:28 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:28.391859 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:28.392841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:28.401712 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:28.402220 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:28.404675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:28.405426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:28.405902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:28.406393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:28.545757 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:28.555592 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:28.557240 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:28.558389 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0f676ca0-be11-48b7-89cb-f5c28e63487b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.173371 Oct 05 14:12:28.558929 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 88/181] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:28 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:29.052049 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:29.053029 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:29.060527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:29.061014 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:29.061445 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:29.062160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:29.062604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:29.063024 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:29.198942 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:29.206941 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c199953b-1283-4101-94a5-0d4f46342d15 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.161347 Oct 05 14:12:29.207515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 94/182] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:29 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:29.285378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:29.286303 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:29.295437 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:29.298042 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:29.298663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:29.299640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:29.300240 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:29.300856 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:29.370582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:29.378469 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4c5cec27-52a2-414a-afbe-b39c46f17e29 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.098475 Oct 05 14:12:29.379071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 89/183] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:29 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:29.587723 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:29.588661 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:29.598830 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:29.600084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:29.600807 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:29.601993 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:29.602703 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:29.603109 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:29.732576 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:29.740692 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:29.742143 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:29.743226 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-48a76363-cb82-4673-8650-43440f87d00b tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.162109 Oct 05 14:12:29.743780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 95/184] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:29 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:30.234543 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:30.235504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:30.246132 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:30.246945 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:30.247658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:30.248792 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:30.249539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:30.250677 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:30.388483 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:30.397538 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-317777d8-156c-4ee5-93b0-29cde6fefe39 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.169950 Oct 05 14:12:30.398743 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 90/185] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:30 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:30.402685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:30.405207 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:30.413335 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:30.413945 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:30.414594 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:30.415316 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:30.425795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:30.426485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:30.489334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:30.500756 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4621f2fb-b38c-4eb7-b1e3-6f8c97b83328 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.102943 Oct 05 14:12:30.501292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 96/186] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:30 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:30.768936 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:30.769903 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:30.777123 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:30.777675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:30.778090 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:30.778832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:30.779279 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:30.779667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:30.913470 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:30.922111 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:30.923572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:30.924670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4916bed7-e3c1-4488-9a4d-8ec2f1a903c8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.160222 Oct 05 14:12:30.925236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 91/187] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:30 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:31.430832 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:31.431616 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:31.438314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:31.438852 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:31.439259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:31.440003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:31.440614 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:31.441149 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:31.524057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:31.524726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:31.544993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:31.545475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:31.545973 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:31.546668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:31.547115 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:31.547538 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:31.576419 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:31.584160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-62312983-4a0a-49b3-8412-97a6832e8149 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.159744 Oct 05 14:12:31.584725 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 97/188] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:31 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:31.610951 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:31.618644 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ff25c5f9-b8be-401e-8734-b3deec24b77c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.098720 Oct 05 14:12:31.619195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 92/189] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:31 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:12:31.948924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:31.949564 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:31.956286 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:31.956772 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:31.957191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:31.957918 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:31.958361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:31.959041 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:32.102491 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:32.114993 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:32.116886 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:32.118431 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f2e33a97-75ab-4967-acf5-1a118dcbd607 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.173576 Oct 05 14:12:32.119036 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 98/190] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:31 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:32.606919 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:32.607554 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:32.613953 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:32.614373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:32.614793 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:32.615467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:32.615906 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:32.616317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:32.640617 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:32.641222 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:32.647205 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:32.647682 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:32.648100 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:32.648778 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:32.649270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:32.649680 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:32.709897 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:32.717916 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-1abe4e99-b951-4d94-b98c-d14b74234ea9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.080977 Oct 05 14:12:32.718524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 99/191] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:32 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:12:32.746607 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:32.758670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c8175176-3520-48c9-ac07-cb98796cdbf7 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.155516 Oct 05 14:12:32.759231 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 93/192] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:32 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:33.146843 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:33.147824 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:33.154881 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:33.155368 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:33.155790 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:33.156467 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:33.156915 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:33.157326 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:33.300017 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:33.312549 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:33.314847 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:33.316522 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e415b651-d84b-4a5f-adcf-bb8d48e8b762 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.176153 Oct 05 14:12:33.317278 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 100/193] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:33 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:33.741414 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:33.742060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:33.748494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:33.748983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:33.749410 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:33.750132 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:33.750578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:33.751001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:33.781773 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:33.782272 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:33.788622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:33.789109 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:33.789532 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:33.790214 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:33.790859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:33.791281 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:33.812493 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:33.820274 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-40f06b73-e309-444f-8872-72775797c532 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.082835 Oct 05 14:12:33.820820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 94/194] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:33 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:33.930415 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:33.938422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-35952b52-4c20-45e1-bcab-5f9b3cc4b13d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.160578 Oct 05 14:12:33.938981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 101/195] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:33 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:34.347472 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:34.348340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:34.358918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:34.359723 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:34.360474 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:34.361606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:34.362388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:34.363081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:34.506799 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:34.515351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:34.517189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:34.517957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-48b00285-5006-46c1-a44a-4930d964624f tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.177198 Oct 05 14:12:34.518495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 95/196] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:34 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:34.847272 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:34.848259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:34.856891 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:34.857366 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:34.857824 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:34.858493 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:34.858934 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:34.859349 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:34.922416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:34.930317 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-83e677db-0d9a-4e98-a0fe-bb3c8714afb2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.089711 Oct 05 14:12:34.930877 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 102/197] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:34 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:34.962039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:34.962690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:34.969017 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:34.969503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:34.969969 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:34.970660 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:34.971100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:34.971534 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:35.101230 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:35.109257 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-27f9d935-34e4-465c-ba0b-3732e98df4bf tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.151226 Oct 05 14:12:35.109855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 96/198] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:34 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:35.543062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:35.543679 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:35.549966 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:35.550455 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:35.550874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:35.551547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:35.551969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:35.552376 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:35.572017 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=100 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:35.680719 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:35.689090 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:35.690543 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:35.691931 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4ef638ad-222f-45d2-9094-aeb668458149 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.152752 Oct 05 14:12:35.692470 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 103/199] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:35 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:35.957429 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:35.958479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:35.966432 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:35.967001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:35.967325 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:35.968010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:35.968454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:35.968869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:36.029367 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:36.037306 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a5393b89-d6cc-4bd9-99a9-4c509ac0f128 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.086648 Oct 05 14:12:36.037929 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 97/200] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:35 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:36.136862 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:36.137921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:36.148087 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:36.148932 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:36.149685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:36.150621 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:36.151065 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:36.151481 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:36.280769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:36.289504 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-5ad080be-a9f4-4055-98ca-f6be7cd4b32e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.160188 Oct 05 14:12:36.290090 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 104/201] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:36 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:36.720500 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:36.721476 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:36.731740 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:36.732529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:36.733706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:36.734831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:36.735573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:36.736251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:36.879836 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:36.888007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:36.889494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:36.890684 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4ac81d6b-385c-4adb-9b1d-3044a651a483 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.176650 Oct 05 14:12:36.891556 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 98/202] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:36 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:37.064191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:37.065175 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:37.074288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:37.074794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:37.075189 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:37.075861 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:37.076304 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:37.076727 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:37.148372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:37.156293 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-60866309-9e03-4e41-82f3-3f5b1e3f904c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.098594 Oct 05 14:12:37.156843 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 105/203] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:37 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:37.312679 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:37.313284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:37.319485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:37.319965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:37.320393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:37.321069 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:37.321517 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:37.321963 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:37.455757 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:37.459231 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:37.464281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9f99b7f5-6407-4aec-8c6f-3673db1a4a19 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.155473 Oct 05 14:12:37.464825 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 99/204] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:37 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:37.916374 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:37.916911 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:37.923466 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:37.923949 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:37.924402 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:37.925090 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:37.925538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:37.925992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:38.057939 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:38.069906 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:38.072193 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:38.073882 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3fd1aca3-91d0-4801-9bf9-ee3ab63399d7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.161468 Oct 05 14:12:38.074599 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 106/205] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:37 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:38.183080 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:38.183711 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:38.191495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:38.192102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:38.192688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:38.193540 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:38.194198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:38.194683 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:38.262221 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:38.270145 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6b863bde-2a51-432a-b8ef-8470ad32342f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1293 microversion: 2.1 time: 0.093581 Oct 05 14:12:38.270700 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 100/206] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:38 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1293 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:38.488773 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:38.489681 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:38.496648 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:38.497127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:38.497549 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:38.498263 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:38.498710 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:38.499122 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:38.644281 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:38.652189 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a28158b0-d642-420f-9ce9-63b83c82d9a8 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.167271 Oct 05 14:12:38.652720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 107/207] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:38 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:39.108377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:39.109616 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:39.115908 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:39.116457 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:39.116915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:39.117812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:39.118478 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:39.119075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:39.293960 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:39.295092 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:39.301333 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:39.301976 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:39.302506 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:39.303433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:39.303852 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:39.304443 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:39.381246 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:39.391018 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-07dcf0b4-5d3d-4348-a036-d3a993949efb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1299 microversion: 2.1 time: 0.101154 Oct 05 14:12:39.402240 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 108/208] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:39 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1299 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:39.542935 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:39.553005 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:39.554396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:39.555735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ee05a143-3fbf-46eb-a976-4ca3bb604ddb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.456237 Oct 05 14:12:39.557321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 101/209] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:39 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:39.677011 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:39.678061 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:39.685157 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:39.686628 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:39.687130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:39.688270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:39.688915 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:39.689432 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.078346 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-f663bbfd-25db-41d7-aed8-fc8eae55d55c req-934506bc-99dc-45fb-b215-285249d9a343 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13f6cef5-9827-4106-afdc-e572e7f742dd", "tag": "f985578d-844d-47e8-8216-66433fdc854b"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:40.090702 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f663bbfd-25db-41d7-aed8-fc8eae55d55c req-934506bc-99dc-45fb-b215-285249d9a343 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.091517 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f663bbfd-25db-41d7-aed8-fc8eae55d55c req-934506bc-99dc-45fb-b215-285249d9a343 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.092319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f663bbfd-25db-41d7-aed8-fc8eae55d55c req-934506bc-99dc-45fb-b215-285249d9a343 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.116380 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-f663bbfd-25db-41d7-aed8-fc8eae55d55c req-934506bc-99dc-45fb-b215-285249d9a343 service nova] Creating event network-changed:f985578d-844d-47e8-8216-66433fdc854b for instance 13f6cef5-9827-4106-afdc-e572e7f742dd on hv-cinder-83587 Oct 05 14:12:40.117538 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f663bbfd-25db-41d7-aed8-fc8eae55d55c req-934506bc-99dc-45fb-b215-285249d9a343 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120605) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 05 14:12:40.118315 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f663bbfd-25db-41d7-aed8-fc8eae55d55c req-934506bc-99dc-45fb-b215-285249d9a343 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120605) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 05 14:12:40.119584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.120245 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.120854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.121787 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.122435 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.123354 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.133253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.compute.rpcapi [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 05 14:12:40.134502 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] Releasing lock "compute-rpcapi-router" {{(pid=120605) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 05 14:12:40.155154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-706c2fcd-d223-47fe-b98d-46ed9b245e79 None None] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.182564 Oct 05 14:12:40.156322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 102/210] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:12:40.163187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:40.176965 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-fc22611d-f9fa-44a5-9521-f2ab23f6f5e1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.504063 Oct 05 14:12:40.190160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 109/211] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:39 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:40.419454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:40.421292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:40.433822 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.434701 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.435514 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.436788 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.437432 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.440375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.500372 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-6e751420-f924-4bf1-ad2d-818c55fb911e req-07246f47-1c89-4a17-aa2b-a0cd4ba60a3e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1439d41a-5fe3-4469-9907-e05aec3ccd59", "tag": "73bbef1c-8f91-49c0-8914-008e1e0d1522"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:40.512056 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-6e751420-f924-4bf1-ad2d-818c55fb911e req-07246f47-1c89-4a17-aa2b-a0cd4ba60a3e service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.512542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-6e751420-f924-4bf1-ad2d-818c55fb911e req-07246f47-1c89-4a17-aa2b-a0cd4ba60a3e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.513696 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-6e751420-f924-4bf1-ad2d-818c55fb911e req-07246f47-1c89-4a17-aa2b-a0cd4ba60a3e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.529483 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-6e751420-f924-4bf1-ad2d-818c55fb911e req-07246f47-1c89-4a17-aa2b-a0cd4ba60a3e service nova] Creating event network-changed:73bbef1c-8f91-49c0-8914-008e1e0d1522 for instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 on hv-cinder-83587 Oct 05 14:12:40.535817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-6e751420-f924-4bf1-ad2d-818c55fb911e req-07246f47-1c89-4a17-aa2b-a0cd4ba60a3e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.150443 Oct 05 14:12:40.537009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 103/212] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:12:40.582393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:40.583100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:40.604424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.604933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.608619 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.609413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:40.609979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:40.610419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:40.695340 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:40.703859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e42b6d6e-39c2-4a04-bb4e-10801244a552 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.291191 Oct 05 14:12:40.704621 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 110/213] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:40 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:12:40.876200 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:40.886820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:40.888840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:40.890566 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fb36c8c5-97c1-4ecb-9a52-7aad75ae3673 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.313630 Oct 05 14:12:40.891789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 104/214] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:40 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:12:41.206641 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:41.211281 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:41.220689 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:41.221251 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:41.221764 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:41.222736 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:41.223917 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:41.224341 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:41.367612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:41.376259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-efbaffc2-4100-46fc-b2e3-b6da46759df1 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.173477 Oct 05 14:12:41.376865 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 111/215] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:41 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:41.729716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:41.730481 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:41.736515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:41.737016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:41.737451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:41.738164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:41.738610 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:41.739025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:41.875793 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:41.883832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-99107c4d-3ccd-45d9-87f3-1f6e36252cf2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.159186 Oct 05 14:12:41.884427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 105/216] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:41 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:41.917031 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:41.917751 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:41.924348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:41.924832 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:41.925270 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:41.925989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:41.926434 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:41.926849 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:42.057438 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:42.065870 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:42.067311 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:42.068397 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-10fb3ec4-d277-4bf9-9b4e-d407c42792b7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.155441 Oct 05 14:12:42.068968 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 112/217] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:41 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:42.400276 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:42.401002 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:42.407923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:42.408410 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:42.408836 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:42.409512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:42.409988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:42.410853 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:42.549827 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:42.560451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-50444afa-e633-450a-8dff-45018ff2969d tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.164448 Oct 05 14:12:42.561193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 106/218] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:42 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:42.570973 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-e4ad74ee-498e-42ae-9315-466fe37e15ec req-f7758f90-23c2-4001-ab47-0c5baee42992 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "26384af3-dd3e-4156-ae91-19e775b78d09", "tag": "9c6d920f-86e6-4f03-ba80-ac14d3dd92bd"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:42.583482 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-e4ad74ee-498e-42ae-9315-466fe37e15ec req-f7758f90-23c2-4001-ab47-0c5baee42992 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:42.583948 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-e4ad74ee-498e-42ae-9315-466fe37e15ec req-f7758f90-23c2-4001-ab47-0c5baee42992 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:42.584367 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-e4ad74ee-498e-42ae-9315-466fe37e15ec req-f7758f90-23c2-4001-ab47-0c5baee42992 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:42.607272 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-e4ad74ee-498e-42ae-9315-466fe37e15ec req-f7758f90-23c2-4001-ab47-0c5baee42992 service nova] Creating event network-changed:9c6d920f-86e6-4f03-ba80-ac14d3dd92bd for instance 26384af3-dd3e-4156-ae91-19e775b78d09 on hv-cinder-83587 Oct 05 14:12:42.607781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-e4ad74ee-498e-42ae-9315-466fe37e15ec req-f7758f90-23c2-4001-ab47-0c5baee42992 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120606) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 05 14:12:42.608160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-e4ad74ee-498e-42ae-9315-466fe37e15ec req-f7758f90-23c2-4001-ab47-0c5baee42992 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120606) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 05 14:12:42.609170 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:42.609657 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:42.610111 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:42.610800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:42.611247 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:42.611644 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:42.621227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.compute.rpcapi [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 05 14:12:42.622097 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] Releasing lock "compute-rpcapi-router" {{(pid=120606) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 05 14:12:42.639132 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8f3950d4-d79c-466d-a5bc-b3661d698082 None None] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071637 Oct 05 14:12:42.639655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 113/219] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:12:42.907578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:42.908240 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:42.915035 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:42.915649 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:42.916074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:42.916797 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:42.917239 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:42.917676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:43.067885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:43.075931 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0f385d67-ea9b-45c5-a51f-709df726efb3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.172379 Oct 05 14:12:43.076485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 107/220] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:42 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:43.093988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:43.094954 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:43.101419 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:43.101938 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:43.102366 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:43.103130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:43.103905 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:43.104356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:43.236123 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:43.244696 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:43.246188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:43.247267 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-200e5a69-ed0a-4e73-8c06-f81ce34bc34c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.157964 Oct 05 14:12:43.247787 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 114/221] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:43 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:43.584074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:43.585503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:43.592813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:43.593596 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:43.594119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:43.594871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:43.595541 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:43.596008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:43.985295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:44.005835 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ca87074c-78c4-4673-9be7-be4a5d4e51ec tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.425449 Oct 05 14:12:44.021368 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 108/222] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:43 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:44.105206 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:44.105922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:44.113005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:44.113971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:44.114598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:44.115490 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:44.116416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:44.117060 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:44.255531 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:44.264084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9b8e13bd-b4f8-4063-aec5-533ccef780b2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.163215 Oct 05 14:12:44.264665 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 115/223] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:44 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:44.283339 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:44.283339 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:44.284517 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:44.285012 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:44.285440 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:44.286167 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:44.286795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:44.287215 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:44.428107 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:44.438449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:44.439899 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:44.441047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-01d21ec7-3727-4096-83db-308b80ded75e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.168048 Oct 05 14:12:44.441612 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 109/224] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:44 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:45.028921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:45.029526 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:45.035920 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:45.036414 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:45.036843 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:45.037529 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:45.038121 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:45.038544 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:45.173349 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:45.181014 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-7f46e39e-5754-4ae1-85c3-f8eca05d341e tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.156452 Oct 05 14:12:45.181580 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 116/225] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:45 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:45.290533 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:45.291491 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:45.298741 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:45.299242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:45.299678 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:45.300507 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:45.300960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:45.301254 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:45.442549 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:45.450716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2db3323a-dcb3-4e6c-91c0-3660a2adae43 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.166382 Oct 05 14:12:45.451311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 110/226] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:45 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:45.466146 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:45.466764 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:45.473144 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:45.473668 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:45.474108 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:45.474797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:45.475251 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:45.475642 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:45.617683 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:45.627250 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:45.628948 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:45.630277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-70e13562-e5e7-44c5-a13c-1312ab51af95 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.168041 Oct 05 14:12:45.631053 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 117/227] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:45 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:46.203809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:46.204414 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:46.210759 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:46.211303 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:46.211732 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:46.212427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:46.212950 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:46.213366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:46.445720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:46.456154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5eefd9b9-4aca-4ede-9a75-dec9e6c20b63 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.256141 Oct 05 14:12:46.456871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 111/228] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:46 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:46.475440 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:46.476680 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:46.485477 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:46.486005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:46.486616 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:46.487312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:46.487759 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:46.488174 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:46.661202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:46.662236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:46.689970 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:46.691143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:46.692362 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:46.693430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:46.694258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:46.694885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:46.777761 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:46.788018 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d5cb2cb6-0abe-4885-b9c1-6adef7ff3ceb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.317915 Oct 05 14:12:46.791853 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 118/229] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:46 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:46.960248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:46.970539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:46.971971 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:46.973108 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ab077f15-e8f3-49e8-bd8b-acf8ad50902e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.321827 Oct 05 14:12:46.973885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 112/230] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:46 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:47.479661 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:47.480308 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:47.486733 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:47.487223 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:47.487681 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:47.488365 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:47.488812 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:47.489223 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:47.619921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:47.628152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4b603576-31ef-45f6-9f1a-56d59b808581 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.152966 Oct 05 14:12:47.628712 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 119/231] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:47 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:47.810985 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:47.811580 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:47.817894 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:47.818382 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:47.818837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:47.819531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:47.819979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:47.820394 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:47.951254 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:47.959766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2dac53fc-2543-4f90-affc-61e70ea3573e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.152699 Oct 05 14:12:47.960359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 113/232] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:47 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:47.998925 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:47.999532 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:48.005949 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:48.006440 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:48.006866 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:48.007570 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:48.008014 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:48.008429 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:48.028053 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=100 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:48.145600 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:48.154675 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:48.156123 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:48.157281 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3e4ce66b-25f4-4fec-83e4-e709a30adc0e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.162510 Oct 05 14:12:48.157841 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 120/233] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:47 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:48.586560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-cef85721-755d-444d-b207-15686b3c9811 req-7c86363f-3efb-40e2-a520-eaa26ec26e1c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13f6cef5-9827-4106-afdc-e572e7f742dd", "name": "network-vif-plugged", "status": "completed", "tag": "f985578d-844d-47e8-8216-66433fdc854b"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:48.596294 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-cef85721-755d-444d-b207-15686b3c9811 req-7c86363f-3efb-40e2-a520-eaa26ec26e1c service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:48.596763 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-cef85721-755d-444d-b207-15686b3c9811 req-7c86363f-3efb-40e2-a520-eaa26ec26e1c service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:48.597180 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-cef85721-755d-444d-b207-15686b3c9811 req-7c86363f-3efb-40e2-a520-eaa26ec26e1c service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:48.610029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-cef85721-755d-444d-b207-15686b3c9811 req-7c86363f-3efb-40e2-a520-eaa26ec26e1c service nova] Creating event network-vif-plugged:f985578d-844d-47e8-8216-66433fdc854b for instance 13f6cef5-9827-4106-afdc-e572e7f742dd on hv-cinder-83587 Oct 05 14:12:48.615086 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-cef85721-755d-444d-b207-15686b3c9811 req-7c86363f-3efb-40e2-a520-eaa26ec26e1c service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091195 Oct 05 14:12:48.616171 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 114/234] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:12:48.651933 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:48.652535 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:48.654491 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=80 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:48.660187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:48.660668 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:48.661103 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:48.661845 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:48.662297 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:48.662718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:48.802688 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:48.810611 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-69eb81f1-93cb-402f-b183-f57190e19613 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.162700 Oct 05 14:12:48.811198 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 121/235] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:48 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:48.985305 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:48.985925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:48.992184 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:48.992660 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:48.993081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:48.993779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:48.994218 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:48.994627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:49.129952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:49.137884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-455a98cc-7154-4bc9-8922-efd494aade47 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.156811 Oct 05 14:12:49.138417 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 115/236] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:48 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:12:49.183357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:49.183966 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:49.190554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:49.191042 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:49.191472 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:49.192162 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:49.192610 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:49.193033 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:49.327168 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:49.356124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:49.357582 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:49.358736 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-891f9f91-8a14-4135-a369-c661ce330654 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.179472 Oct 05 14:12:49.359287 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 122/237] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:49 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:49.837708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:49.838894 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:49.849041 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:49.849872 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:49.850591 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:49.851705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:49.852444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:49.853521 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:49.995549 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:50.003524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a3e9b0d6-c043-4b4d-b2fe-7cb8aed8f4bd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.172288 Oct 05 14:12:50.004248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 116/238] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:49 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:50.161968 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:50.163062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:50.172284 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:50.173016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:50.173878 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:50.175063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:50.175880 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:50.176524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:50.312462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:50.320379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-65fe1372-3c82-46e0-b93c-acaf62221653 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.163704 Oct 05 14:12:50.320955 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 123/239] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:50 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:50.384667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:50.385248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:50.388114 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:50.394600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:50.395090 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:50.395518 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:50.396543 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:50.397138 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:50.397735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:50.529023 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:50.537961 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:50.539423 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:50.540587 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c56687a9-784f-47fc-8458-c4757c8a7d0c tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.159929 Oct 05 14:12:50.541158 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 117/240] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:50 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:50.638247 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-480d8e41-6739-4c4d-8aa1-13d03f6a1168 req-71513b24-9afb-4ad7-a3f1-3a2bd22de237 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1439d41a-5fe3-4469-9907-e05aec3ccd59", "name": "network-vif-plugged", "status": "completed", "tag": "73bbef1c-8f91-49c0-8914-008e1e0d1522"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:50.647990 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-480d8e41-6739-4c4d-8aa1-13d03f6a1168 req-71513b24-9afb-4ad7-a3f1-3a2bd22de237 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:50.648484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-480d8e41-6739-4c4d-8aa1-13d03f6a1168 req-71513b24-9afb-4ad7-a3f1-3a2bd22de237 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:50.648892 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-480d8e41-6739-4c4d-8aa1-13d03f6a1168 req-71513b24-9afb-4ad7-a3f1-3a2bd22de237 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:50.666337 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-480d8e41-6739-4c4d-8aa1-13d03f6a1168 req-71513b24-9afb-4ad7-a3f1-3a2bd22de237 service nova] Creating event network-vif-plugged:73bbef1c-8f91-49c0-8914-008e1e0d1522 for instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 on hv-cinder-83587 Oct 05 14:12:50.672036 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-480d8e41-6739-4c4d-8aa1-13d03f6a1168 req-71513b24-9afb-4ad7-a3f1-3a2bd22de237 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040871 Oct 05 14:12:50.672634 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 124/241] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:12:51.028387 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:51.029316 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:51.035845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:51.036412 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:51.036853 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:51.037556 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:51.038041 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:51.038443 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:51.042688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:51.173297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:51.181438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1a666174-2ba9-4706-84c5-e999ec884c4c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.158236 Oct 05 14:12:51.182054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 118/242] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:51 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:51.344396 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:51.345004 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:51.351736 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:51.352218 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:51.352651 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:51.353665 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:51.354147 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:51.354567 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:51.490285 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:51.500396 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-cbaeaa94-0d76-4494-8584-4a1ffe49b1de tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.160406 Oct 05 14:12:51.513802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 125/243] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:51 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:51.566241 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:51.566858 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:51.573001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:51.573476 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:51.573919 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:51.574592 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:51.575057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:51.575468 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:51.720796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:51.730131 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:51.732019 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:51.733146 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5b496485-4968-45b2-a7a0-9a7c53eea6d0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1404 microversion: 2.20 time: 0.171043 Oct 05 14:12:51.733732 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 119/244] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:51 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1404 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:52.204662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:52.205233 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:52.211640 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.212373 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.212835 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:52.213570 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.217608 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.217608 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:52.352899 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:52.361462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f26dc7b1-e7f1-4c49-a82d-59c9b87c058c tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1315 microversion: 2.1 time: 0.160745 Oct 05 14:12:52.362055 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 126/245] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:52 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:52.528938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:52.533422 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:52.541236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=82 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:52.544324 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.544998 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.545433 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:52.546217 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.546717 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.547140 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:52.692113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:52.700053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0b840dbc-bbb7-471e-b3c7-39ddf309c08e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.179419 Oct 05 14:12:52.700601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 120/246] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:52 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:52.758583 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:52.759391 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:52.761473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:52.767910 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.768097 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.768643 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:52.769860 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.769975 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.770452 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:52.914646 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:52.922999 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:52.924473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:52.925627 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f2a745ed-a1e4-41b8-ac72-16a3940354cf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1596 microversion: 2.20 time: 0.171118 Oct 05 14:12:52.926224 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 127/247] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:12:52 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1596 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:12:52.950865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:52.951351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:52.957461 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.957978 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.958404 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:52.959079 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:52.959525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:52.959933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:53.096227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b9067b3a-eb64-45fd-b7fc-b16bf844f9a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.149434 Oct 05 14:12:53.096847 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 121/248] 10.222.0.23 () {60 vars in 1316 bytes} [Wed Oct 5 14:12:52 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd/os-interface => generated 284 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:12:53.386594 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:53.387414 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:53.394343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:53.395025 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:53.395463 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:53.396188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:53.396718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:53.397080 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:53.723578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:53.724216 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:53.730681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:53.731315 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:53.731749 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:53.732439 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:53.732890 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:53.733309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:53.825949 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:53.849211 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-aa8ba216-7565-4952-989c-0098252cbdbe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1502 microversion: 2.1 time: 0.466007 Oct 05 14:12:53.882518 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 128/249] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:53 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1502 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:53.900852 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:53.916367 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b98e7e6a-51d6-4fc7-96b4-9e29385e7980 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.196586 Oct 05 14:12:53.917102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 122/250] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:53 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:12:54.177893 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-381ad532-56d1-423e-83ac-2f154d6cafc5 req-89787818-0726-49c2-88dd-30efcdf51e1b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "26384af3-dd3e-4156-ae91-19e775b78d09", "name": "network-vif-plugged", "status": "completed", "tag": "9c6d920f-86e6-4f03-ba80-ac14d3dd92bd"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:54.188943 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-381ad532-56d1-423e-83ac-2f154d6cafc5 req-89787818-0726-49c2-88dd-30efcdf51e1b service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:54.189754 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-381ad532-56d1-423e-83ac-2f154d6cafc5 req-89787818-0726-49c2-88dd-30efcdf51e1b service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:54.190746 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-381ad532-56d1-423e-83ac-2f154d6cafc5 req-89787818-0726-49c2-88dd-30efcdf51e1b service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:54.203882 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-381ad532-56d1-423e-83ac-2f154d6cafc5 req-89787818-0726-49c2-88dd-30efcdf51e1b service nova] Creating event network-vif-plugged:9c6d920f-86e6-4f03-ba80-ac14d3dd92bd for instance 26384af3-dd3e-4156-ae91-19e775b78d09 on hv-cinder-83587 Oct 05 14:12:54.209122 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-381ad532-56d1-423e-83ac-2f154d6cafc5 req-89787818-0726-49c2-88dd-30efcdf51e1b service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034843 Oct 05 14:12:54.210447 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 129/251] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:12:54.777176 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-7ced1872-4081-46d1-96c2-8d6a96ea71b1 req-3d494b57-73d4-4adb-a4c0-b2cb68f736d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13f6cef5-9827-4106-afdc-e572e7f742dd", "tag": "f985578d-844d-47e8-8216-66433fdc854b"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:54.786933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-7ced1872-4081-46d1-96c2-8d6a96ea71b1 req-3d494b57-73d4-4adb-a4c0-b2cb68f736d2 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:54.787410 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-7ced1872-4081-46d1-96c2-8d6a96ea71b1 req-3d494b57-73d4-4adb-a4c0-b2cb68f736d2 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:54.788024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-7ced1872-4081-46d1-96c2-8d6a96ea71b1 req-3d494b57-73d4-4adb-a4c0-b2cb68f736d2 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:54.800306 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-7ced1872-4081-46d1-96c2-8d6a96ea71b1 req-3d494b57-73d4-4adb-a4c0-b2cb68f736d2 service nova] Creating event network-changed:f985578d-844d-47e8-8216-66433fdc854b for instance 13f6cef5-9827-4106-afdc-e572e7f742dd on hv-cinder-83587 Oct 05 14:12:54.805697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-7ced1872-4081-46d1-96c2-8d6a96ea71b1 req-3d494b57-73d4-4adb-a4c0-b2cb68f736d2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032138 Oct 05 14:12:54.806531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 123/252] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:12:54.940549 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:54.941541 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:54.948275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:54.948757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:54.949181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:54.950007 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:54.950511 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:54.950982 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:55.125178 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:55.133114 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-edac893a-29d8-4138-80e7-3bbd8f4b2eef tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.197686 Oct 05 14:12:55.134137 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 130/253] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:54 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:56.157358 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:56.158084 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:56.164822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:56.165317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:56.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:56.166479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:56.166924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:56.167363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:56.300489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:56.308221 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9580c47c-0c04-4140-8f88-2501793aec0a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1366 microversion: 2.1 time: 0.155300 Oct 05 14:12:56.308783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 124/254] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:56 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:57.335545 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:57.336441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:57.344421 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:57.344937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:57.345378 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:57.346104 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:57.346586 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:57.347001 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:57.484598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:12:57.492639 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-fa9e3801-29bb-45fd-9f0d-821bed1b479e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1556 microversion: 2.1 time: 0.163831 Oct 05 14:12:57.493400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 131/255] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:12:57 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1556 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:12:57.515523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:12:57.516026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:12:57.522105 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:57.522577 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:57.522996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:57.523665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:57.524099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:57.524506 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:57.601829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-917caeb3-b1b6-460f-8931-18d0743f0085 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.088790 Oct 05 14:12:57.601829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 125/256] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 5 14:12:57 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09/os-interface => generated 284 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:12:58.552189 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-5b67c194-a492-46f7-b5b4-0447977333fc req-916e5670-4e4c-433c-9a3a-ea1bd6b520a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "26384af3-dd3e-4156-ae91-19e775b78d09", "tag": "9c6d920f-86e6-4f03-ba80-ac14d3dd92bd"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:12:58.561914 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-5b67c194-a492-46f7-b5b4-0447977333fc req-916e5670-4e4c-433c-9a3a-ea1bd6b520a4 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:12:58.562383 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-5b67c194-a492-46f7-b5b4-0447977333fc req-916e5670-4e4c-433c-9a3a-ea1bd6b520a4 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:12:58.562794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-5b67c194-a492-46f7-b5b4-0447977333fc req-916e5670-4e4c-433c-9a3a-ea1bd6b520a4 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:12:58.574185 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-5b67c194-a492-46f7-b5b4-0447977333fc req-916e5670-4e4c-433c-9a3a-ea1bd6b520a4 service nova] Creating event network-changed:9c6d920f-86e6-4f03-ba80-ac14d3dd92bd for instance 26384af3-dd3e-4156-ae91-19e775b78d09 on hv-cinder-83587 Oct 05 14:12:58.579531 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-5b67c194-a492-46f7-b5b4-0447977333fc req-916e5670-4e4c-433c-9a3a-ea1bd6b520a4 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032149 Oct 05 14:12:58.580224 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 132/257] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:12:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:14:20.852693 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=11 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:14:20.853990 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=38 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:14:20.862453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:14:20.863249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:14:20.870412 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:14:20.870884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:14:20.871353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:14:20.872040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:14:20.872482 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:14:20.872890 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:14:20.897618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ddf70b76-25d1-4103-a7c7-9834a7d3ec64 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09/ips" status: 200 len: 149 microversion: 2.1 time: 0.048629 Oct 05 14:14:20.898224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 126/258] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:14:20 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09/ips => generated 149 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:14:24.726294 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=23 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:14:24.727373 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=62 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:14:24.733101 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:14:24.733699 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:14:24.740827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:14:24.741306 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:14:24.741766 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:14:24.742448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:14:24.742896 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:14:24.743314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:14:24.803605 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-437c3460-0718-45bf-a5c6-d15e9738477d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd/ips" status: 200 len: 152 microversion: 2.20 time: 0.079282 Oct 05 14:14:24.804165 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 133/259] 10.222.0.23 () {60 vars in 1289 bytes} [Wed Oct 5 14:14:24 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd/ips => generated 152 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:20:21.302312 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:21.303101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:21.304272 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:20:21.304492 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:20:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 (ip 10.222.0.23) !!! Oct 05 14:20:21.305509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:20:21.305708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:20:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 (ip 10.222.0.23) !!! Oct 05 14:20:21.306597 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:20:21.306741 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:20:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 (ip 10.222.0.23) !!! Oct 05 14:20:21.310992 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:21.311550 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:21.323382 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:21.323856 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:21.324282 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:21.325132 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:21.325408 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:21.325857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:21.352986 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Going to try to terminate instance {{(pid=120605) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:20:21.464529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-bd7eb315-a9f3-47c5-8f17-331969fee4f2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 204 len: 0 microversion: 2.1 time: 0.165628 Oct 05 14:20:21.468916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 127/260] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:20:21 2022] DELETE /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:20:21.476741 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:21.477624 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=4 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:21.482225 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:20:21.482541 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:20:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 (ip 10.222.0.23) !!! Oct 05 14:20:21.484094 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:20:21.484436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:20:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 (ip 10.222.0.23) !!! Oct 05 14:20:21.486058 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:20:21.486311 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:20:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 (ip 10.222.0.23) !!! Oct 05 14:20:21.498853 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:21.500002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:21.522048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:21.522830 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:21.523380 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:21.524186 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:21.524731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:21.525245 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:21.887831 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:21.895989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d4963f23-e491-4996-bdee-65e7f0a96e49 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1680 microversion: 2.1 time: 0.425613 Oct 05 14:20:21.896850 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 134/261] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:21 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1680 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:22.924582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:22.925482 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:22.932448 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:22.933206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:22.934097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:22.935057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:22.935685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:22.936327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:23.132026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:23.140201 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2428b496-07ad-4d4c-a7e1-4cb18e0de589 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1613 microversion: 2.1 time: 0.219742 Oct 05 14:20:23.141136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 128/262] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:22 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1613 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:23.465066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-7871329b-8ce5-4c7f-8cf5-ebe05596c317 req-cfb2859f-bd89-48b3-8919-b96dbd0b1db9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "26384af3-dd3e-4156-ae91-19e775b78d09", "name": "network-vif-deleted", "tag": "9c6d920f-86e6-4f03-ba80-ac14d3dd92bd"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:23.477785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-7871329b-8ce5-4c7f-8cf5-ebe05596c317 req-cfb2859f-bd89-48b3-8919-b96dbd0b1db9 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:23.478619 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-7871329b-8ce5-4c7f-8cf5-ebe05596c317 req-cfb2859f-bd89-48b3-8919-b96dbd0b1db9 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:23.480695 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-7871329b-8ce5-4c7f-8cf5-ebe05596c317 req-cfb2859f-bd89-48b3-8919-b96dbd0b1db9 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:23.493140 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-7871329b-8ce5-4c7f-8cf5-ebe05596c317 req-cfb2859f-bd89-48b3-8919-b96dbd0b1db9 service nova] Creating event network-vif-deleted:9c6d920f-86e6-4f03-ba80-ac14d3dd92bd for instance 26384af3-dd3e-4156-ae91-19e775b78d09 on hv-cinder-83587 Oct 05 14:20:23.499862 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-7871329b-8ce5-4c7f-8cf5-ebe05596c317 req-cfb2859f-bd89-48b3-8919-b96dbd0b1db9 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122928 Oct 05 14:20:23.501343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 135/263] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:24.164071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:24.165568 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:24.172253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:24.173009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:24.173704 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:24.174412 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:24.174916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:24.175372 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:24.210504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:24.210890 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:24.211313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:24.211776 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lazy-loading 'fault' on Instance uuid 26384af3-dd3e-4156-ae91-19e775b78d09 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:20:24.257902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:24.265849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-33c9d7f3-ece8-471b-8d3e-87c7dcee0269 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 200 len: 1331 microversion: 2.1 time: 0.105500 Oct 05 14:20:24.266316 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 129/264] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:24 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 1331 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:25.134334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:25.134890 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:25.140966 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:25.141444 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:25.141905 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:25.142616 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:25.143065 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:25.143486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:25.168423 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:20:25.264737 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-7d810a21-9cb8-4a6b-8ce1-c9fa4317ada3 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "DELETE /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 204 len: 0 microversion: 2.20 time: 0.134256 Oct 05 14:20:25.265294 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 136/265] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 5 14:20:25 2022] DELETE /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 05 14:20:25.273794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:25.274399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:25.280397 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:25.280871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:25.281293 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:25.282569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:25.283011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:25.283762 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:25.289490 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:25.289854 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:25.296130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:25.296598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:25.297040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:25.297784 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:25.298384 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:25.298803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:25.313705 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 HTTP/1.1 Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:20:25.318375 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:20:25.320795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '26384af3-dd3e-4156-ae91-19e775b78d09'} Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:20:25.322152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:20:25.324259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:20:25.328325 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:20:25.328325 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4b739fab-9372-4b8a-b813-8c5bbaaa4290 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 404 len: 111 microversion: 2.1 time: 0.038660 Oct 05 14:20:25.328325 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 137/266] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:25 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 05 14:20:25.334601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e7e49685-7f7f-423e-99d1-e738567854e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '.version_select at 0x7f77c99a39d0>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:25.363334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e7e49685-7f7f-423e-99d1-e738567854e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1649165095" status: 202 len: 0 microversion: 2.1 time: 0.032592 Oct 05 14:20:25.364698 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 138/267] 10.222.0.23 () {58 vars in 1221 bytes} [Wed Oct 5 14:20:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1649165095 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 05 14:20:25.473097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:25.481521 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:25.482997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:25.484171 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-dd2dc518-111c-4ad6-9ca2-02486cbc16c5 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1721 microversion: 2.20 time: 0.214293 Oct 05 14:20:25.484735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 130/268] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:25 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:26.071391 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-1d57deaf-3170-4c25-8236-14d04c71ec97 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Action: 'create', calling method: .version_select at 0x7f77c99b2f70>, body: {"keypair": {"name": "tempest-keypair-637087651", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIs1cVOm+OpL6A0+2NSaTkaulu8+JJFvJvlCa2dc8hRusK9mcEaONm/aO8u7XJhxS033o3U/0hW9MU87HzyiaFtolBLxKrj5HhFvG1hSMqYLa4/9b4XIiqndflmDZwglJQ=="}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:26.089359 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-1d57deaf-3170-4c25-8236-14d04c71ec97 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'key_pairs'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:20:26.093266 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-1d57deaf-3170-4c25-8236-14d04c71ec97 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for user 2e988e0435464c1d8aa4842fab56dfa4 and project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'key_pairs'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:20:26.138607 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-1d57deaf-3170-4c25-8236-14d04c71ec97 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.070978 Oct 05 14:20:26.141686 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 139/269] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:20:26 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:26.510474 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:26.511821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:26.518897 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:26.519577 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:26.520007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:26.520906 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:26.521354 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:26.522009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:26.778731 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:26.791114 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:26.792918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:26.794278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-05be624b-4b76-4f24-a360-6eb4d90bf03d tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1721 microversion: 2.20 time: 0.288073 Oct 05 14:20:26.795346 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 131/270] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:26 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1721 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:27.256432 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-62b9e824-d9b4-467a-9d22-d523d3448803 req-a04625af-f332-46f5-94a7-ac0347c5e3aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13f6cef5-9827-4106-afdc-e572e7f742dd", "name": "network-vif-deleted", "tag": "f985578d-844d-47e8-8216-66433fdc854b"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:27.269543 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-62b9e824-d9b4-467a-9d22-d523d3448803 req-a04625af-f332-46f5-94a7-ac0347c5e3aa service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:27.270422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-62b9e824-d9b4-467a-9d22-d523d3448803 req-a04625af-f332-46f5-94a7-ac0347c5e3aa service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:27.271210 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-62b9e824-d9b4-467a-9d22-d523d3448803 req-a04625af-f332-46f5-94a7-ac0347c5e3aa service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:27.288733 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-62b9e824-d9b4-467a-9d22-d523d3448803 req-a04625af-f332-46f5-94a7-ac0347c5e3aa service nova] Creating event network-vif-deleted:f985578d-844d-47e8-8216-66433fdc854b for instance 13f6cef5-9827-4106-afdc-e572e7f742dd on hv-cinder-83587 Oct 05 14:20:27.294810 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-62b9e824-d9b4-467a-9d22-d523d3448803 req-a04625af-f332-46f5-94a7-ac0347c5e3aa service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042282 Oct 05 14:20:27.298648 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 140/271] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:27.750520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1693445261", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b6b92f47-a2f9-4f0e-8749-335f8828b651"}], "security_groups": [{"name": "tempest-securitygroup--829802864"}], "key_name": "tempest-keypair-637087651", "user_data": "}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:27.820082 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:27.829611 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:27.838524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:27.839360 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:27.839968 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:27.840823 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:27.841598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:27.842197 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:27.875214 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.network.neutron [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] validate_networks() for [('b6b92f47-a2f9-4f0e-8749-335f8828b651', None, None, None, None, None)] {{(pid=120605) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:20:27.878871 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:27.879535 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:27.880517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:27.881666 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.objects.instance [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lazy-loading 'fault' on Instance uuid 13f6cef5-9827-4106-afdc-e572e7f742dd {{(pid=120606) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:20:27.954388 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:27.969128 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:27.971163 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:27.974037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-466b976c-594b-45dc-b6c0-c4f800221a09 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 200 len: 1371 microversion: 2.20 time: 0.157427 Oct 05 14:20:27.975137 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 141/272] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:27 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 1371 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:20:28.403299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Flavor limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:20:28.403855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Image limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:20:28.404134 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Flavor pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:20:28.404529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Image pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:20:28.404966 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:20:28.413864 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:28.414295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:28.414713 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:28.415411 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:28.415844 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:28.416247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:28.432822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:20:28.436657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for user 2e988e0435464c1d8aa4842fab56dfa4 and project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:20:28.447902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Going to run 1 instances... {{(pid=120605) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:20:28.450202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:20:28.500013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b0e466d5-094d-4c5c-92ea-9c2c08f5cb0d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.755852 Oct 05 14:20:28.500888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 132/273] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:20:27 2022] POST /compute/v2.1/servers => generated 401 bytes in 757 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:20:28.512968 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0432e7e6-d495-48c5-8cdc-812ab3d7d01a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:28.512968 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0432e7e6-d495-48c5-8cdc-812ab3d7d01a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:28.571923 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0432e7e6-d495-48c5-8cdc-812ab3d7d01a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:28.579628 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0432e7e6-d495-48c5-8cdc-812ab3d7d01a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1241 microversion: 2.1 time: 0.071134 Oct 05 14:20:28.580515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 142/274] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:28 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1241 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:28.999165 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:28.999927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:29.006818 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:29.007640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:29.008052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:29.008933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:29.009503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:29.009928 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:29.026615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] HTTP exception thrown: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Request method failure captured: Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd HTTP/1.1 Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCIMQVjIopH9J5k4QVCgWqfjnjjnB1gb3lTFqFSV1Bz3OG_QdqDm9-htnPLVrDJYsQHKuvH1VTk6k_hVoMJ7YFtDEnGfOMmCvcvB9_LnJ6fWE_7-LqLbJGn5007TuAXc9zrEDsif-LnJ_z6KZ09Q-u1bdydicPrxSvD0DJZdBktU6s Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Openstack-Nova-Api-Version: 2.20 Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:20:29.031489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_6bd3e20c75e94f3a9c17fcb3029bfdff", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bad6116fbd5c407c9f44084fadd9531e Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:20:29.033119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '13f6cef5-9827-4106-afdc-e572e7f742dd'} Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:20:29.033880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:20:29.035049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:20:29.036033 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Returning 404 to user: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:20:29.036033 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-576291ac-cf5d-471e-b83f-090e618bd0a7 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 404 len: 111 microversion: 2.20 time: 0.037766 Oct 05 14:20:29.036033 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 133/275] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:28 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 05 14:20:29.047362 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-04baf6b2-70da-420a-a2ad-8e9334159e11 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '.version_select at 0x7f77c99b2790>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:29.062482 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-04baf6b2-70da-420a-a2ad-8e9334159e11 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-936939098" status: 204 len: 0 microversion: 2.20 time: 0.023242 Oct 05 14:20:29.063252 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 143/276] 10.222.0.23 () {60 vars in 1259 bytes} [Wed Oct 5 14:20:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-936939098 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 05 14:20:29.602818 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:29.603421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:29.611096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:29.611575 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:29.611983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:29.612685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:29.613118 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:29.613529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:29.685395 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:29.694916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ad52f8f0-4ed4-4cc4-89fd-65b547219230 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1293 microversion: 2.1 time: 0.096128 Oct 05 14:20:29.696165 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 134/277] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:29 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1293 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:29.748585 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-ac008b78-edca-44e8-b79a-b396f15135fa tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Action: 'create', calling method: .version_select at 0x7f77c99a3c10>, body: {"keypair": {"name": "tempest-keypair-140775366", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBNnjgUrH0kv2fG2J4tvQ6Uny2owzjUw89iHZWKMsqsCJ20VSYUG/eY9CYIwg2PSDgZSOA4ustYZdwG7B1BUiH9S5Ohb2rsEXi18Vs3+Wf3wg2otujnxkLCQUgELX6Plng=="}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:29.766427 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-ac008b78-edca-44e8-b79a-b396f15135fa tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'key_pairs'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:20:29.770442 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-ac008b78-edca-44e8-b79a-b396f15135fa tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for user bad6116fbd5c407c9f44084fadd9531e and project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'key_pairs'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:20:29.794206 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-ac008b78-edca-44e8-b79a-b396f15135fa tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.049579 Oct 05 14:20:29.795248 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 144/278] 10.222.0.23 () {62 vars in 1200 bytes} [Wed Oct 5 14:20:29 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 51 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 05 14:20:30.720430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:30.721864 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:30.735954 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:30.737177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:30.737842 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:30.738781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:30.739875 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:30.740602 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:30.991625 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:31.001804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4590bd90-a137-40df-9446-dabcf66ae815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.285591 Oct 05 14:20:31.008040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 135/279] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:30 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:31.151874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1980663153", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "58657efd-d41f-4391-a8ec-e6996bf42b73"}], "security_groups": [{"name": "tempest-securitygroup--1370552198"}], "key_name": "tempest-keypair-140775366", "user_data": "}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:31.264188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.network.neutron [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] validate_networks() for [('58657efd-d41f-4391-a8ec-e6996bf42b73', None, None, None, None, None)] {{(pid=120606) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:20:31.368832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-b08d2caa-0b99-44b9-9a60-fdd94684cbe8 req-ea0cb344-1304-4c28-b347-e618c1da5119 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "41b4c31d-52e5-4475-94d1-d54892780716", "tag": "42cca083-7f47-4f75-a81b-db9d9cef539c"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:31.380121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b08d2caa-0b99-44b9-9a60-fdd94684cbe8 req-ea0cb344-1304-4c28-b347-e618c1da5119 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:31.380600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b08d2caa-0b99-44b9-9a60-fdd94684cbe8 req-ea0cb344-1304-4c28-b347-e618c1da5119 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:31.381054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b08d2caa-0b99-44b9-9a60-fdd94684cbe8 req-ea0cb344-1304-4c28-b347-e618c1da5119 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:31.392933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-b08d2caa-0b99-44b9-9a60-fdd94684cbe8 req-ea0cb344-1304-4c28-b347-e618c1da5119 service nova] Creating event network-changed:42cca083-7f47-4f75-a81b-db9d9cef539c for instance 41b4c31d-52e5-4475-94d1-d54892780716 on hv-cinder-83587 Oct 05 14:20:31.398175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-b08d2caa-0b99-44b9-9a60-fdd94684cbe8 req-ea0cb344-1304-4c28-b347-e618c1da5119 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033496 Oct 05 14:20:31.399316 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 136/280] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:20:31.710036 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Flavor limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:20:31.710457 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Image limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:20:31.710924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Flavor pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:20:31.711325 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Image pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:20:31.711727 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:20:31.721109 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:31.721597 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:31.722115 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:31.722844 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:31.723288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:31.723703 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:31.740556 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:20:31.744631 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Getting quotas for user bad6116fbd5c407c9f44084fadd9531e and project 6bd3e20c75e94f3a9c17fcb3029bfdff. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:20:31.755761 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Going to run 1 instances... {{(pid=120606) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:20:31.758181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120606) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:20:31.814337 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a1cc86d4-1663-487f-955d-7845f56f00fe tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.667441 Oct 05 14:20:31.814990 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 145/281] 10.222.0.23 () {62 vars in 1188 bytes} [Wed Oct 5 14:20:31 2022] POST /compute/v2.1/servers => generated 402 bytes in 669 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 05 14:20:31.827099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3868236b-ce53-4d53-80eb-63d5c12905b8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:31.827708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3868236b-ce53-4d53-80eb-63d5c12905b8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:31.895813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3868236b-ce53-4d53-80eb-63d5c12905b8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:31.904146 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3868236b-ce53-4d53-80eb-63d5c12905b8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:31.905584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3868236b-ce53-4d53-80eb-63d5c12905b8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:31.906987 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3868236b-ce53-4d53-80eb-63d5c12905b8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1281 microversion: 2.20 time: 0.084301 Oct 05 14:20:31.907525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 137/282] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:31 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1281 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:32.025169 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:32.025827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:32.032959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:32.033500 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:32.034015 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:32.034757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:32.035275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:32.035754 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:32.182884 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:32.191261 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-73890e16-a739-446c-8a5d-5abf874bafd0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.170320 Oct 05 14:20:32.193694 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 146/283] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:32 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:32.933732 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:32.934334 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:32.941096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:32.941580 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:32.942050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:32.942735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:32.943175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:32.943589 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:33.004617 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:33.013141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:33.014600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:33.015684 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3e25389e-c21a-4005-bf19-4cefb6006750 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1285 microversion: 2.20 time: 0.085987 Oct 05 14:20:33.016243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 138/284] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:32 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1285 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:33.220233 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:33.221111 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:33.229769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:33.230237 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:33.230676 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:33.231373 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:33.231824 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:33.232229 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:33.388438 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:33.402306 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f7c27a7b-e1d1-45fc-8824-596688b7beba tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.189552 Oct 05 14:20:33.403128 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 147/285] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:33 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:34.040957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:34.041900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:34.048565 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:34.049192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:34.049735 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:34.050795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:34.051315 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:34.051786 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:34.142884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:34.153038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:34.154600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:34.155826 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8b7318e0-57d1-4b68-be76-1101ed3e7aab tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1277 microversion: 2.20 time: 0.119153 Oct 05 14:20:34.156672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 139/286] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:34 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1277 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:20:34.428155 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:34.428805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:34.435096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:34.435585 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:34.436012 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:34.436701 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:34.437146 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:34.437564 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:34.626352 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:34.648129 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c6db7afe-bf18-4839-ab16-02a9bcb23a84 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.223928 Oct 05 14:20:34.652484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 148/287] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:34 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:35.182947 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:35.184351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:35.191783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:35.192316 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:35.192800 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:35.193758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:35.194211 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:35.194811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:35.259094 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:35.267745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:35.269191 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:35.270347 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2e3012d4-12ec-41cc-a9d2-a69316c219c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1339 microversion: 2.20 time: 0.092751 Oct 05 14:20:35.270963 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 140/288] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:35 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1339 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:35.679510 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:35.680143 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:35.686941 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:35.687430 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:35.687857 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:35.688549 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:35.689008 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:35.689499 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:35.924239 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:35.932358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e5b6da2c-5640-49f6-8ae5-0cf91244ffa6 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.257281 Oct 05 14:20:35.932926 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 149/289] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:35 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:36.296244 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:36.296867 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:36.303037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:36.303522 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:36.303924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:36.304780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:36.305398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:36.305834 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:36.322083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-97bddb5a-9215-4035-90b1-6e0f7cd3ea6b req-a6dbaaa5-11d4-4027-b9bb-87fc2d4616c8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "172fdb3d-4bc0-4955-afed-d2cfe04ce532", "tag": "040d3759-79ba-48f0-bf0a-1092f124ad90"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:36.333091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-97bddb5a-9215-4035-90b1-6e0f7cd3ea6b req-a6dbaaa5-11d4-4027-b9bb-87fc2d4616c8 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:36.333562 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-97bddb5a-9215-4035-90b1-6e0f7cd3ea6b req-a6dbaaa5-11d4-4027-b9bb-87fc2d4616c8 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:36.334052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-97bddb5a-9215-4035-90b1-6e0f7cd3ea6b req-a6dbaaa5-11d4-4027-b9bb-87fc2d4616c8 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:36.345869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-97bddb5a-9215-4035-90b1-6e0f7cd3ea6b req-a6dbaaa5-11d4-4027-b9bb-87fc2d4616c8 service nova] Creating event network-changed:040d3759-79ba-48f0-bf0a-1092f124ad90 for instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 on hv-cinder-83587 Oct 05 14:20:36.354008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-97bddb5a-9215-4035-90b1-6e0f7cd3ea6b req-a6dbaaa5-11d4-4027-b9bb-87fc2d4616c8 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106206 Oct 05 14:20:36.354635 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 141/290] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:36.523278 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:36.548874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:36.550350 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:36.551454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-555b7e82-bc13-4e56-8460-16eaa3e50c62 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1407 microversion: 2.20 time: 0.259516 Oct 05 14:20:36.552229 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 150/291] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:36 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1407 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:36.697223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-e15824ce-54cf-41af-b72c-baf049624cff req-e2a380a8-54e7-4211-875e-1bc63ab3c4dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "41b4c31d-52e5-4475-94d1-d54892780716", "name": "network-vif-plugged", "status": "completed", "tag": "42cca083-7f47-4f75-a81b-db9d9cef539c"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:36.715437 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-e15824ce-54cf-41af-b72c-baf049624cff req-e2a380a8-54e7-4211-875e-1bc63ab3c4dc service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:36.716016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-e15824ce-54cf-41af-b72c-baf049624cff req-e2a380a8-54e7-4211-875e-1bc63ab3c4dc service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:36.716441 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-e15824ce-54cf-41af-b72c-baf049624cff req-e2a380a8-54e7-4211-875e-1bc63ab3c4dc service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:36.731994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-e15824ce-54cf-41af-b72c-baf049624cff req-e2a380a8-54e7-4211-875e-1bc63ab3c4dc service nova] Creating event network-vif-plugged:42cca083-7f47-4f75-a81b-db9d9cef539c for instance 41b4c31d-52e5-4475-94d1-d54892780716 on hv-cinder-83587 Oct 05 14:20:36.739764 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-e15824ce-54cf-41af-b72c-baf049624cff req-e2a380a8-54e7-4211-875e-1bc63ab3c4dc service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107091 Oct 05 14:20:36.740553 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 142/292] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:36.959749 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:36.960886 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:36.971373 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:36.972179 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:36.972893 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:36.974085 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:36.974878 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:36.975295 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:37.151629 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:37.163079 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d8c96d72-97b4-474b-9262-e2e509ae64cb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.210023 Oct 05 14:20:37.163649 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 151/293] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:36 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:37.576814 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:37.577492 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:37.584550 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:37.585041 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:37.585445 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:37.586353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:37.586783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:37.587399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:37.725196 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:37.733536 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:37.735005 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:37.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f9fb2274-66b5-4b7d-995a-a793bf7d31ad tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1407 microversion: 2.20 time: 0.163607 Oct 05 14:20:37.736671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 143/294] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:37 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:38.188534 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:38.189427 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:38.199617 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:38.200311 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:38.200918 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:38.201949 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:38.202592 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:38.203187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:38.336886 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:38.345156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9a7ea714-d0bc-43ff-bbe5-f1a3bc24c981 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.162165 Oct 05 14:20:38.345720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 152/295] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:38 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:38.775666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:38.776645 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:38.779777 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:38.788480 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:38.788964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:38.789391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:38.790104 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:38.790572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:38.790960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:38.920856 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:38.929469 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:38.930946 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:38.932028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-282a5a3d-8d26-4b27-b8b3-e713a25bbef6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1407 microversion: 2.20 time: 0.162989 Oct 05 14:20:38.933370 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 144/296] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:38 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:39.372893 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:39.374449 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:39.383575 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:39.384060 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:39.384489 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:39.385181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:39.385649 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:39.386053 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:39.530770 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:39.538812 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a3698ce6-5b66-402d-bd31-922e89a3887e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.172805 Oct 05 14:20:39.551705 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 153/297] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:39 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:39.959056 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:39.959970 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:39.969361 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:39.970083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:39.970692 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:39.971658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:39.972293 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:39.972885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:40.115484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:40.120736 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:40.127563 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:40.129019 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:40.130165 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3f8d675f-a626-4d40-bee6-ef5bb9e2f6e6 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1407 microversion: 2.20 time: 0.176949 Oct 05 14:20:40.130731 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 145/298] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:39 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:40.561788 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:40.562403 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:40.569072 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:40.569809 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:40.570292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:40.571066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:40.572610 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:40.572610 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:40.704294 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:40.712957 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b59c61cd-15d6-4176-b32e-1bcf6d187997 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1366 microversion: 2.1 time: 0.155218 Oct 05 14:20:40.713538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 154/299] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:40 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:41.156503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:41.157287 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:41.168427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:41.169565 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:41.170026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:41.170731 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:41.171173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:41.171583 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:41.311665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:41.323491 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:41.325170 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:41.326529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ad2ef28b-f6ce-4e76-a4e8-cd9ad7d347a8 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1407 microversion: 2.20 time: 0.173979 Oct 05 14:20:41.327286 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 146/300] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:41 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1407 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:41.741400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:41.742316 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:41.744291 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:41.750054 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:41.750528 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:41.750953 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:41.751623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:41.752066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:41.752536 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:41.753057 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=16 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:41.888094 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:41.896229 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0e24d67b-01c2-42db-bc3c-b0732a231118 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1556 microversion: 2.1 time: 0.162527 Oct 05 14:20:41.896813 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 155/301] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:20:41 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1556 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:20:41.920912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:41.921626 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:41.924391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=8 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:41.930752 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:41.931224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:41.931646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:41.932321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:41.932761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:41.933168 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:42.028122 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c3ba2bdb-14d9-4c10-8a87-8bb6e9b78e40 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.113082 Oct 05 14:20:42.028676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 147/302] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 5 14:20:41 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716/os-interface => generated 284 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:42.352473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:42.353232 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:42.359826 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:42.360413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:42.360847 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:42.361548 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:42.362030 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:42.362449 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:42.366888 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:42.632194 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-97913948-71cf-4876-b49b-ae646f1faee8 req-ca7564f0-13c5-46ea-aa96-f34941ab7a1c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "172fdb3d-4bc0-4955-afed-d2cfe04ce532", "name": "network-vif-plugged", "status": "completed", "tag": "040d3759-79ba-48f0-bf0a-1092f124ad90"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:42.649544 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-97913948-71cf-4876-b49b-ae646f1faee8 req-ca7564f0-13c5-46ea-aa96-f34941ab7a1c service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:42.650766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-97913948-71cf-4876-b49b-ae646f1faee8 req-ca7564f0-13c5-46ea-aa96-f34941ab7a1c service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:42.656740 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-97913948-71cf-4876-b49b-ae646f1faee8 req-ca7564f0-13c5-46ea-aa96-f34941ab7a1c service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:42.671807 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-97913948-71cf-4876-b49b-ae646f1faee8 req-ca7564f0-13c5-46ea-aa96-f34941ab7a1c service nova] Creating event network-vif-plugged:040d3759-79ba-48f0-bf0a-1092f124ad90 for instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 on hv-cinder-83587 Oct 05 14:20:42.677554 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-97913948-71cf-4876-b49b-ae646f1faee8 req-ca7564f0-13c5-46ea-aa96-f34941ab7a1c service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048945 Oct 05 14:20:42.678907 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 148/303] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:42.734908 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:42.745003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:42.746568 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:42.747797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-186b8350-3465-4482-ae79-56cad0496422 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1407 microversion: 2.20 time: 0.399495 Oct 05 14:20:42.761816 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 156/304] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:42 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1407 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:43.342835 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-b16cda2a-2ef6-441a-a27a-efd064effe21 req-f4edc928-9e7a-4a45-a81b-bd40f80fc8c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "41b4c31d-52e5-4475-94d1-d54892780716", "tag": "42cca083-7f47-4f75-a81b-db9d9cef539c"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:43.354227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b16cda2a-2ef6-441a-a27a-efd064effe21 req-f4edc928-9e7a-4a45-a81b-bd40f80fc8c5 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:43.354733 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b16cda2a-2ef6-441a-a27a-efd064effe21 req-f4edc928-9e7a-4a45-a81b-bd40f80fc8c5 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:43.355155 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b16cda2a-2ef6-441a-a27a-efd064effe21 req-f4edc928-9e7a-4a45-a81b-bd40f80fc8c5 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:43.367125 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-b16cda2a-2ef6-441a-a27a-efd064effe21 req-f4edc928-9e7a-4a45-a81b-bd40f80fc8c5 service nova] Creating event network-changed:42cca083-7f47-4f75-a81b-db9d9cef539c for instance 41b4c31d-52e5-4475-94d1-d54892780716 on hv-cinder-83587 Oct 05 14:20:43.372594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-b16cda2a-2ef6-441a-a27a-efd064effe21 req-f4edc928-9e7a-4a45-a81b-bd40f80fc8c5 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033939 Oct 05 14:20:43.373177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 149/305] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:20:43.772732 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:43.773365 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:43.779652 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:43.780136 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:43.780560 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:43.781244 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:43.781722 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:43.782143 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:43.926413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:43.948989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:43.948989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:43.948989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e3ca1ec4-a6b2-4a1b-8189-639ce6f6887e tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1407 microversion: 2.20 time: 0.175850 Oct 05 14:20:43.949279 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 157/306] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:43 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1407 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:44.972993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:44.974351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:44.980747 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:44.981231 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:44.981710 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:44.982450 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:44.983229 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:44.983683 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:45.117624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:45.126864 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:45.128296 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:20:45.129429 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-7a2b63d7-376c-4f86-a0af-99afe1d704c1 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1600 microversion: 2.20 time: 0.164154 Oct 05 14:20:45.130001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 150/307] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:20:44 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1600 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:20:45.153797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:20:45.154364 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:20:45.157197 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:DELETE=2,nova_api:INSERT=5 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:45.160902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:45.161383 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:45.161852 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:45.162540 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:45.162989 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:45.163400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:45.253466 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-643aa6f2-d3f9-40f2-9bc9-9681501e9be9 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.103567 Oct 05 14:20:45.254160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 158/308] 10.222.0.23 () {60 vars in 1316 bytes} [Wed Oct 5 14:20:45 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532/os-interface => generated 285 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:20:46.459269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-64e55efa-a10c-4dd5-b7e3-0739f98db3fa req-1029cdf9-0a08-47d5-9579-153577526777 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "172fdb3d-4bc0-4955-afed-d2cfe04ce532", "tag": "040d3759-79ba-48f0-bf0a-1092f124ad90"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:20:46.469010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-64e55efa-a10c-4dd5-b7e3-0739f98db3fa req-1029cdf9-0a08-47d5-9579-153577526777 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:20:46.469482 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-64e55efa-a10c-4dd5-b7e3-0739f98db3fa req-1029cdf9-0a08-47d5-9579-153577526777 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:20:46.469989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-64e55efa-a10c-4dd5-b7e3-0739f98db3fa req-1029cdf9-0a08-47d5-9579-153577526777 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:20:46.480737 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-64e55efa-a10c-4dd5-b7e3-0739f98db3fa req-1029cdf9-0a08-47d5-9579-153577526777 service nova] Creating event network-changed:040d3759-79ba-48f0-bf0a-1092f124ad90 for instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 on hv-cinder-83587 Oct 05 14:20:46.485624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-64e55efa-a10c-4dd5-b7e3-0739f98db3fa req-1029cdf9-0a08-47d5-9579-153577526777 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030005 Oct 05 14:20:46.486241 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 151/309] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:20:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:21:58.383618 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=14 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:21:58.472950 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:21:58.473677 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:21:58.481555 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:21:58.482071 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:21:58.482505 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:21:58.483192 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:21:58.483641 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:21:58.484058 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:21:58.507125 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2ef9395e-3ef7-4049-a913-cf48a3f708e2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716/ips" status: 200 len: 148 microversion: 2.1 time: 0.124771 Oct 05 14:21:58.507863 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 159/310] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:21:58 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716/ips => generated 148 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:22:01.375795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:22:01.376582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=22 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:22:01.471561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:22:01.472786 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:22:01.479874 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:22:01.480305 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:22:01.480708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:22:01.481396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:22:01.481853 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:22:01.482464 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:22:01.511615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ac698224-1148-4a0f-b4ad-3691785bd678 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532/ips" status: 200 len: 153 microversion: 2.20 time: 0.137467 Oct 05 14:22:01.513020 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 152/311] 10.222.0.23 () {60 vars in 1289 bytes} [Wed Oct 5 14:22:01 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532/ips => generated 153 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 05 14:24:54.200958 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:24:54.201566 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:24:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 (ip 10.222.0.23) !!! Oct 05 14:24:54.203182 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:24:54.203329 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:24:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 (ip 10.222.0.23) !!! Oct 05 14:24:54.205002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:24:54.205112 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:24:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 (ip 10.222.0.23) !!! Oct 05 14:24:54.206214 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:24:54.206946 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=4 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:24:54.212143 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:24:54.212622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:24:54.228694 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:54.229167 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:54.229593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:54.230300 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:54.230743 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:54.231151 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:54.257861 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:24:54.348835 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-cb0b41de-7efd-407f-9ea1-20a53a7b19fe tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "DELETE /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 204 len: 0 microversion: 2.1 time: 0.149423 Oct 05 14:24:54.349496 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 160/312] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:24:54 2022] DELETE /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:24:54.355519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:24:54.356271 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:24:54.357045 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:24:54.357196 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:24:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 (ip 10.222.0.23) !!! Oct 05 14:24:54.358318 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:24:54.358448 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:24:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 (ip 10.222.0.23) !!! Oct 05 14:24:54.359281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:24:54.359558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:24:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 (ip 10.222.0.23) !!! Oct 05 14:24:54.366331 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:24:54.366938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:24:54.378454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:54.378949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:54.379366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:54.380051 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:54.380500 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:54.380921 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:54.516150 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:24:54.524218 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-87f49b89-7344-464b-81e0-b63a45e9ab58 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1508 microversion: 2.1 time: 0.170212 Oct 05 14:24:54.524785 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 153/313] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:24:54 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1508 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:24:55.548875 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:24:55.549661 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:24:55.555896 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:55.556443 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:55.556937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:55.557672 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:55.558237 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:55.558711 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:55.803140 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '596b52fe5595438d8acd593c61c91f67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '04395d53d99840c3b16d49e2f6d62631', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:24:55.811453 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9e75089d-1947-4e25-b654-293530511509 tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 200 len: 1508 microversion: 2.1 time: 0.267044 Oct 05 14:24:55.812184 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 161/314] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:24:55 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 1508 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:24:55.833199 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-51901410-008a-4bd8-8dba-11b3b2e227d8 req-25c81d0e-9422-4b32-906b-446e9766cc4e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1439d41a-5fe3-4469-9907-e05aec3ccd59", "name": "network-vif-deleted", "tag": "73bbef1c-8f91-49c0-8914-008e1e0d1522"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:24:55.844057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-51901410-008a-4bd8-8dba-11b3b2e227d8 req-25c81d0e-9422-4b32-906b-446e9766cc4e service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:55.844829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-51901410-008a-4bd8-8dba-11b3b2e227d8 req-25c81d0e-9422-4b32-906b-446e9766cc4e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:55.844944 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-51901410-008a-4bd8-8dba-11b3b2e227d8 req-25c81d0e-9422-4b32-906b-446e9766cc4e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:55.857269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-51901410-008a-4bd8-8dba-11b3b2e227d8 req-25c81d0e-9422-4b32-906b-446e9766cc4e service nova] Creating event network-vif-deleted:73bbef1c-8f91-49c0-8914-008e1e0d1522 for instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 on hv-cinder-83587 Oct 05 14:24:55.870623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-51901410-008a-4bd8-8dba-11b3b2e227d8 req-25c81d0e-9422-4b32-906b-446e9766cc4e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042011 Oct 05 14:24:55.871742 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 154/315] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:24:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:24:56.836814 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:24:56.837526 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] [instance: 1439d41a-5fe3-4469-9907-e05aec3ccd59] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:24:56.844866 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:56.845804 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:56.846322 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:56.847463 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:24:56.847978 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:24:56.848459 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:24:56.863854 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] HTTP exception thrown: Instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 could not be found. Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Request method failure captured: Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 HTTP/1.1 Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZCNeaZd0AcIhQ6JFsr8F9c1PyCv9O3vxvlKWeb9wPdfgC0cmn0no6IDnOOR2-i1Thw2c6tuCJz2_grAtS3KjajlbU5r_IH8paMNAhECaEmkxKoD3LxHB5T1gxaFqqZl1vt4TRLo6WmSYK4buMe2h8I28qFMF3Zum0jkGdzEI7HCzTw Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 04395d53d99840c3b16d49e2f6d62631 Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-DeleteServersTestJSON-563896428 Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:24:56.865515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_04395d53d99840c3b16d49e2f6d62631", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/04395d53d99840c3b16d49e2f6d62631"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/04395d53d99840c3b16d49e2f6d62631"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/04395d53d99840c3b16d49e2f6d62631"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-DeleteServersTestJSON-563896428 Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 04395d53d99840c3b16d49e2f6d62631 Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-DeleteServersTestJSON-563896428 Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-DeleteServersTestJSON-563896428-project Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 596b52fe5595438d8acd593c61c91f67 Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-DeleteServersTestJSON-563896428-project Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 could not be found. Oct 05 14:24:56.867161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '1439d41a-5fe3-4469-9907-e05aec3ccd59'} Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 could not be found. Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:24:56.868228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 could not be found. Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 could not be found. Oct 05 14:24:56.869720 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:24:56.870980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] Returning 404 to user: Instance 1439d41a-5fe3-4469-9907-e05aec3ccd59 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:24:56.870980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6622da79-dd3f-4291-99b7-1c6a9bb67ccd tempest-DeleteServersTestJSON-563896428 tempest-DeleteServersTestJSON-563896428-project] 10.222.0.23 "GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59" status: 404 len: 111 microversion: 2.1 time: 0.034664 Oct 05 14:24:56.870980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 162/316] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:24:56 2022] GET /compute/v2.1/servers/1439d41a-5fe3-4469-9907-e05aec3ccd59 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:25:05.960442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:05.961617 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=8 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:06.082000 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-141885436", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:25:06.139030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.network.neutron [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] validate_networks() for [('0fb733ad-3060-4312-be4b-61d6c0a01518', None, None, None, None, None)] {{(pid=120605) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:25:06.579589 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Flavor limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:25:06.579999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Image limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:25:06.580462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Flavor pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:25:06.580858 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Image pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:25:06.581258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:25:06.590114 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:06.590589 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:06.591006 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:06.591716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:06.592157 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:06.592573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:06.610283 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Getting quotas for project ae3d0e9a5de8481fa19a58f7780a4b7a. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:25:06.614292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Getting quotas for user 6e4a13a6981f4e0898dc22ea8525a1ee and project ae3d0e9a5de8481fa19a58f7780a4b7a. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:25:06.625436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Going to run 1 instances... {{(pid=120605) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:25:06.628102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:25:06.682172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-619c8095-a644-47c2-8c06-5b14634a0af1 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 0.722536 Oct 05 14:25:06.683015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 155/317] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:25:05 2022] POST /compute/v2.1/servers => generated 383 bytes in 724 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:25:06.695217 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-22878c6c-7c10-4c51-8428-b36d06ea7fc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:06.695840 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-22878c6c-7c10-4c51-8428-b36d06ea7fc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:06.761003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-22878c6c-7c10-4c51-8428-b36d06ea7fc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:06.769677 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-22878c6c-7c10-4c51-8428-b36d06ea7fc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1" status: 200 len: 1221 microversion: 2.1 time: 0.078893 Oct 05 14:25:06.770824 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 163/318] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:06 2022] GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1 => generated 1221 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:25:06.844462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-272662917", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:25:06.935820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.network.neutron [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] validate_networks() for [('0fb733ad-3060-4312-be4b-61d6c0a01518', None, None, None, None, None)] {{(pid=120605) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:25:07.386023 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Flavor limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:25:07.386706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Image limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:25:07.386920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Flavor pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:25:07.387525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Image pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:25:07.388973 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:25:07.400202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:07.400773 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:07.401286 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:07.402301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:07.402869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:07.403477 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:07.421394 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Getting quotas for project ae3d0e9a5de8481fa19a58f7780a4b7a. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:25:07.425568 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Getting quotas for user 6e4a13a6981f4e0898dc22ea8525a1ee and project ae3d0e9a5de8481fa19a58f7780a4b7a. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:25:07.437017 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Going to run 1 instances... {{(pid=120605) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:25:07.439330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:25:07.492016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a2711244-4c9f-42f2-9ae4-7bd23ea5fdd5 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 0.652078 Oct 05 14:25:07.493047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 156/319] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:25:06 2022] POST /compute/v2.1/servers => generated 383 bytes in 654 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:25:07.504297 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-39c1d487-a535-4e2e-9275-94de9691547b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:07.505094 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-39c1d487-a535-4e2e-9275-94de9691547b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:07.510855 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=22,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:07.568623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-39c1d487-a535-4e2e-9275-94de9691547b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:07.578798 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-39c1d487-a535-4e2e-9275-94de9691547b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1221 microversion: 2.1 time: 0.078868 Oct 05 14:25:07.579709 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 164/320] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:07 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1221 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:25:08.602829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:08.603517 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:08.611014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:08.611743 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:08.612169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:08.613052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:08.613667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:08.614239 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:08.679229 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:08.687180 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-434f9cf4-3419-4f17-bd70-d28cd83259fd tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1273 microversion: 2.1 time: 0.088615 Oct 05 14:25:08.687923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 157/321] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:08 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1273 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:09.710232 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:09.710845 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:09.717001 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:09.717485 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:09.717954 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:09.718865 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:09.719472 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:09.720124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:09.892173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-3dc9da03-b2b4-4b02-a881-17bf5b6d8a6a req-11ef1618-8155-4c86-be20-6e4543ccc461 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "451ebffc-7a95-491f-9c6b-6f508bc584d1", "tag": "b772ad13-fc5f-48d5-b449-22494f53ecf8"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:25:09.922206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-3dc9da03-b2b4-4b02-a881-17bf5b6d8a6a req-11ef1618-8155-4c86-be20-6e4543ccc461 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:09.923236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-3dc9da03-b2b4-4b02-a881-17bf5b6d8a6a req-11ef1618-8155-4c86-be20-6e4543ccc461 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:09.925311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-3dc9da03-b2b4-4b02-a881-17bf5b6d8a6a req-11ef1618-8155-4c86-be20-6e4543ccc461 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:09.947715 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-3dc9da03-b2b4-4b02-a881-17bf5b6d8a6a req-11ef1618-8155-4c86-be20-6e4543ccc461 service nova] Creating event network-changed:b772ad13-fc5f-48d5-b449-22494f53ecf8 for instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 on hv-cinder-83587 Oct 05 14:25:09.963110 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-3dc9da03-b2b4-4b02-a881-17bf5b6d8a6a req-11ef1618-8155-4c86-be20-6e4543ccc461 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074986 Oct 05 14:25:09.965579 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 158/322] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:25:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:25:10.025988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:10.038546 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-847ac2e1-360d-4b2f-8581-5bd8f288155d tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.332051 Oct 05 14:25:10.040273 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 165/323] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:09 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:11.013346 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-bd7d4fec-135a-4902-9ecc-5220ca7ce242 req-d731b435-58c1-4d3c-aab8-581e79f94ad4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db6f4760-f8d9-4962-bdb0-623a7f51267c", "tag": "47e256dc-16ba-45a9-a0db-f7e74740b0b0"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:25:11.023130 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-bd7d4fec-135a-4902-9ecc-5220ca7ce242 req-d731b435-58c1-4d3c-aab8-581e79f94ad4 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:11.023849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-bd7d4fec-135a-4902-9ecc-5220ca7ce242 req-d731b435-58c1-4d3c-aab8-581e79f94ad4 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:11.024509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-bd7d4fec-135a-4902-9ecc-5220ca7ce242 req-d731b435-58c1-4d3c-aab8-581e79f94ad4 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:11.036296 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-bd7d4fec-135a-4902-9ecc-5220ca7ce242 req-d731b435-58c1-4d3c-aab8-581e79f94ad4 service nova] Creating event network-changed:47e256dc-16ba-45a9-a0db-f7e74740b0b0 for instance db6f4760-f8d9-4962-bdb0-623a7f51267c on hv-cinder-83587 Oct 05 14:25:11.042126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-bd7d4fec-135a-4902-9ecc-5220ca7ce242 req-d731b435-58c1-4d3c-aab8-581e79f94ad4 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032408 Oct 05 14:25:11.042740 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 159/324] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:25:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:25:11.066302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:11.066908 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:11.073432 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:11.073939 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:11.074373 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:11.075065 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:11.075504 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:11.075919 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:11.210032 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:11.217924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-929af5e7-b866-4dfd-85e5-547abfc5cf15 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.156718 Oct 05 14:25:11.218596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 166/325] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:11 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:12.242301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:12.243040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:12.250327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:12.251040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:12.251681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:12.252668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:12.253352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:12.254016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:12.399198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:12.408753 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-275bab81-7d0e-4d95-9929-598ab8d0a125 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.170909 Oct 05 14:25:12.409411 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 160/326] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:12 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:13.432413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:13.433068 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:13.449969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:13.449969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:13.449969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:13.450666 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:13.451106 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:13.451514 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:13.701278 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:13.709320 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-dd573fcc-6d44-4f6e-97b6-e0952f717c0c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.280729 Oct 05 14:25:13.709927 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 167/327] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:13 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:14.733113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:14.733773 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:14.740224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:14.740702 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:14.741127 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:14.741851 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:14.742295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:14.742705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:14.886926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:14.894831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c5590604-6724-4534-821a-9622dc9aac1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.165710 Oct 05 14:25:14.895395 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 161/328] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:14 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:15.917803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:15.918578 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:15.925288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:15.925817 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:15.926242 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:15.926929 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:15.927374 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:15.927778 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:16.072513 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:16.082251 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4fb842ea-fe19-4e6f-8f1f-22a121677d76 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.168372 Oct 05 14:25:16.083523 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 168/329] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:15 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:16.095483 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-4971a883-4020-4254-9533-853e4741f009 req-b29062b8-687f-48bd-adaf-9757e8d7b24d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "451ebffc-7a95-491f-9c6b-6f508bc584d1", "name": "network-vif-plugged", "status": "completed", "tag": "b772ad13-fc5f-48d5-b449-22494f53ecf8"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:25:16.105416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-4971a883-4020-4254-9533-853e4741f009 req-b29062b8-687f-48bd-adaf-9757e8d7b24d service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:16.105917 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-4971a883-4020-4254-9533-853e4741f009 req-b29062b8-687f-48bd-adaf-9757e8d7b24d service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:16.106337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-4971a883-4020-4254-9533-853e4741f009 req-b29062b8-687f-48bd-adaf-9757e8d7b24d service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:16.121333 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-4971a883-4020-4254-9533-853e4741f009 req-b29062b8-687f-48bd-adaf-9757e8d7b24d service nova] Creating event network-vif-plugged:b772ad13-fc5f-48d5-b449-22494f53ecf8 for instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 on hv-cinder-83587 Oct 05 14:25:16.126836 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-4971a883-4020-4254-9533-853e4741f009 req-b29062b8-687f-48bd-adaf-9757e8d7b24d service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035216 Oct 05 14:25:16.127636 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 162/330] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:25:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:25:17.106276 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:17.106895 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:17.113383 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:17.114056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:17.114468 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:17.115149 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:17.115600 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:17.116017 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:17.399946 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:17.407753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8acb60b1-596e-43b3-836b-443983204d43 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.305661 Oct 05 14:25:17.408366 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 169/331] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:17 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:18.153816 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:18.154525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=12 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:18.158826 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-6ae69655-d159-44c8-a4a3-ac352c86d6ad req-0f8e2954-33c8-4b98-bc0c-79ccdd30abf5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db6f4760-f8d9-4962-bdb0-623a7f51267c", "name": "network-vif-plugged", "status": "completed", "tag": "47e256dc-16ba-45a9-a0db-f7e74740b0b0"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:25:18.169500 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-6ae69655-d159-44c8-a4a3-ac352c86d6ad req-0f8e2954-33c8-4b98-bc0c-79ccdd30abf5 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:18.170007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-6ae69655-d159-44c8-a4a3-ac352c86d6ad req-0f8e2954-33c8-4b98-bc0c-79ccdd30abf5 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:18.170428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-6ae69655-d159-44c8-a4a3-ac352c86d6ad req-0f8e2954-33c8-4b98-bc0c-79ccdd30abf5 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:18.182360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-6ae69655-d159-44c8-a4a3-ac352c86d6ad req-0f8e2954-33c8-4b98-bc0c-79ccdd30abf5 service nova] Creating event network-vif-plugged:47e256dc-16ba-45a9-a0db-f7e74740b0b0 for instance db6f4760-f8d9-4962-bdb0-623a7f51267c on hv-cinder-83587 Oct 05 14:25:18.187334 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-6ae69655-d159-44c8-a4a3-ac352c86d6ad req-0f8e2954-33c8-4b98-bc0c-79ccdd30abf5 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034902 Oct 05 14:25:18.187950 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 163/332] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:25:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:25:18.430406 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:18.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:18.433756 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=4 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:18.439766 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:18.441729 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:18.442319 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:18.442979 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:18.443445 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:18.443838 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:18.647912 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:18.655562 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6c570137-58e5-46c2-a7fa-96f9a690ea6b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.229131 Oct 05 14:25:18.656107 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 170/333] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:18 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:19.683476 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:19.684622 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:19.693202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:19.693742 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:19.694144 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:19.694817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:19.695464 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:19.695884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:19.827759 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:19.836036 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a10cc47a-822f-439c-89ef-51ebd662248a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1321 microversion: 2.1 time: 0.159792 Oct 05 14:25:19.836643 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 164/334] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:19 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1321 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:20.859797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:20.860574 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:20.867229 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:20.867746 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:20.868261 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:20.868981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:20.869484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:20.869929 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:21.053137 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:21.061201 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-45b89258-0663-46fd-b9ea-e4e8769396f3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1515 microversion: 2.1 time: 0.205438 Oct 05 14:25:21.061920 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 171/335] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:20 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1515 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:21.084333 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:25:21.086858 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:21.093070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:21.093670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:21.094079 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:21.094760 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:21.095198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:21.095606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:21.119179 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lazy-loading 'flavor' on Instance uuid db6f4760-f8d9-4962-bdb0-623a7f51267c {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:25:21.235587 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ba0a4959-5c18-4762-91dd-320356af8086 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "POST /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c/action" status: 200 len: 32 microversion: 2.1 time: 0.155321 Oct 05 14:25:21.236685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 165/336] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 5 14:25:21 2022] POST /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c/action => generated 32 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 05 14:25:21.246608 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:21.247220 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:21.254344 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:21.254795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:21.255226 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:21.255924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:21.256416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:21.256835 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:21.467869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:21.490746 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3659e5bd-c606-4793-927a-ac743fcc90fc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.247791 Oct 05 14:25:21.491320 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 172/337] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:21 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:22.514920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:22.515811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:22.522624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:22.523215 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:22.523722 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:22.524549 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:22.525079 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:22.525573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:22.663512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:22.675755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ad4ddaa3-601a-432c-a6de-d33cc3493025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.164973 Oct 05 14:25:22.676561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 166/338] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:22 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:23.703083 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:23.704056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:23.712028 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:23.712517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:23.712946 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:23.713666 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:23.714127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:23.714547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:23.847992 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:23.859521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3f42eb84-acae-4088-81b2-597f559cedaa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.162635 Oct 05 14:25:23.860282 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 173/339] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:23 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:24.886477 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:24.887335 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:24.897696 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:24.898555 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:24.899254 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:24.900376 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:24.901114 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:24.901846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:25.099761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:25.107321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5ebb2bd7-89f7-42c0-9179-791dcfa26775 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.227612 Oct 05 14:25:25.107870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 167/340] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:24 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:26.130581 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:26.131132 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:26.133029 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:26.138180 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:26.138666 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:26.139097 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:26.139782 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:26.140228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:26.140644 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:26.274784 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:26.283171 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-acb4d9fe-129c-46f6-b719-270f1d494a1a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.156572 Oct 05 14:25:26.283729 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 174/341] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:26 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:27.307358 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:27.308095 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:27.316175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:27.316811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:27.317379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:27.318327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:27.319015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:27.319559 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:27.455629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:27.463428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e2c69673-374c-4dc8-9be1-57a587b72e88 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.160989 Oct 05 14:25:27.464002 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 168/342] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:27 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:28.487187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:28.487775 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:28.494440 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:28.496492 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:28.496938 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:28.497615 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:28.498090 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:28.498511 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:28.730372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:28.738882 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-7e72fbf9-31c1-44f7-9854-7cd29b5f2c51 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.255627 Oct 05 14:25:28.739453 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 175/343] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:28 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:29.762761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:29.763387 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:29.769770 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:29.770248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:29.770695 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:29.771345 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:29.771790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:29.772298 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:30.051405 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:30.059227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-adc12280-649b-4eff-88a0-14ec111ed025 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.300351 Oct 05 14:25:30.059796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 169/344] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:29 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:31.083360 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:31.083924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:31.090538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:31.091022 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:31.091453 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:31.092136 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:31.092585 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:31.093003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:31.238063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:31.246222 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-502b6840-2edb-4a55-9f20-a9ce5ec2ef22 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.167033 Oct 05 14:25:31.246807 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 176/345] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:31 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:32.273417 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:32.274471 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:32.284427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:32.285148 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:32.285988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:32.287041 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:32.287720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:32.288378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:32.422976 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:32.430854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-abe2464a-ef3d-48cf-882f-1fc3348f589a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.163868 Oct 05 14:25:32.437153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 170/346] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:32 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:33.462304 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:33.463331 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:33.469960 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:33.470450 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:33.470889 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:33.471612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:33.472059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:33.472817 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:33.617715 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:33.632503 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-023c1d90-dfc1-45ab-87a6-d3b864077ae3 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.177569 Oct 05 14:25:33.633064 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 177/347] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:33 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:34.657476 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:34.658208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:34.664265 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:34.664744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:34.665164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:34.665897 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:34.666338 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:34.666746 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:34.802087 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:34.809899 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a4d04f3f-c3cc-40d1-92cf-ea8ae97cc94a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.156249 Oct 05 14:25:34.810478 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 171/348] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:34 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:35.837153 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:35.838145 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:35.848544 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:35.849338 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:35.850106 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:35.851256 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:35.852007 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:35.852711 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:35.985953 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:35.997740 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-89097258-7f84-4daf-b077-c0a4c26ef79b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.166721 Oct 05 14:25:35.998581 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 178/349] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:35 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:37.029250 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:37.029862 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:37.032016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=6 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:37.038322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:37.038889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:37.039357 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:37.040088 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:37.040561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:37.040999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:37.044371 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:37.172708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:37.184232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-bd24ebea-8a3e-49c1-ab5c-a65ee02a90cc tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.160306 Oct 05 14:25:37.184997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 172/350] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:37 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:38.217556 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:38.218646 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:38.229135 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:38.229962 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:38.230674 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:38.231983 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:38.232455 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:38.232877 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:38.235459 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=85 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:38.380115 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:38.390550 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d675ff79-5d15-4057-b7c0-3a2deed14070 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.179392 Oct 05 14:25:38.391591 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 179/351] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:38 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:39.419539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:39.420301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:39.428169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:39.428771 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:39.429301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:39.430208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:39.430767 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:39.431285 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:39.572984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:39.580870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-473082fd-bf10-4624-9832-1b3c557b18fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.168090 Oct 05 14:25:39.581438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 173/352] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:39 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:40.607784 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:40.608470 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:40.615097 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:40.615605 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:40.616025 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:40.616689 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:40.617126 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:40.617582 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:40.747612 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:40.755518 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-820f7d80-9f0a-420a-b7dd-d4f862826fef tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.151606 Oct 05 14:25:40.756084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 180/353] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:40 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:41.786308 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:41.787238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:41.797555 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:41.798437 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:41.799163 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:41.800309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:41.801065 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:41.801830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:41.937597 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:41.945416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-437d8733-c787-4b65-8210-637e3226bce9 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.166341 Oct 05 14:25:41.953851 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 174/354] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:41 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:42.973073 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:42.974114 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:42.984135 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:42.984622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:42.985052 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:42.985769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:42.986260 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:42.986644 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:43.138757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:43.149674 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2d033a60-6c75-48d6-9b0b-8985efce40f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.183569 Oct 05 14:25:43.150230 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 181/355] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:42 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:44.172529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:44.173153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:44.179137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:44.179798 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:44.180231 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:44.180921 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:44.181368 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:44.181813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:44.310823 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:44.319182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-bde43767-22b2-4504-a044-2441deb6cf48 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.150592 Oct 05 14:25:44.320110 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 175/356] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:44 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:45.352380 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:45.353401 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:45.362089 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:45.362577 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:45.363003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:45.363688 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:45.364141 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:45.364565 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:45.495447 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:45.503696 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-98f70394-f318-4ddd-a20d-ef2700a89a7a tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.158119 Oct 05 14:25:45.504602 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 182/357] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:45 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:46.529290 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:46.530228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:46.537047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:46.537530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:46.537998 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:46.538723 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:46.539167 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:46.539583 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:46.670816 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:46.678899 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-acf07100-1b08-4092-adcf-3d87bcb7541c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.155809 Oct 05 14:25:46.679438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 176/358] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:46 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:25:47.702239 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:47.702921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:47.709167 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:47.709665 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:47.710107 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:47.710785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:47.711227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:47.711661 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:47.842376 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:47.850819 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6cb21931-59e2-4419-966e-42f0e0213270 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.152854 Oct 05 14:25:47.851382 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 183/359] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:47 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:48.876282 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:48.877239 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:48.887815 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:48.888632 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:48.889350 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:48.890755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:48.891244 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:48.891665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:49.038331 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:49.046350 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-477056ba-1ca5-4857-a7cb-b4bc1170a434 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.175786 Oct 05 14:25:49.046932 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 177/360] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:48 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:50.070123 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:50.070693 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:50.077123 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:50.077610 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:50.078064 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:50.078754 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:50.079186 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:50.079600 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:50.217310 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:50.225858 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-718a84fc-94ff-4a08-b7ee-2748e82d6e5c tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.159570 Oct 05 14:25:50.226411 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 184/361] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:50 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:51.248281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:51.248879 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:51.254934 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:51.255506 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:51.255942 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:51.256623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:51.257061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:51.257470 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:51.389063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:51.396889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c670eb2a-5865-4e84-90af-a4561a58b009 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.152423 Oct 05 14:25:51.397755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 178/362] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:51 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:52.424700 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:52.425736 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:52.436285 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:52.436999 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:52.437721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:52.438908 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:52.439685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:52.440105 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:52.575323 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:52.584040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-bd8afd56-dc1d-4f6a-94c7-ee79f77fe916 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.166038 Oct 05 14:25:52.584707 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 185/363] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:52 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:53.610855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:53.611545 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:53.618105 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:53.618751 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:53.618997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:53.619709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:53.620146 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:53.620556 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:53.767661 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:53.775502 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-498ac2c0-0343-4af2-af8a-707e3c7f2dc7 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.171318 Oct 05 14:25:53.776092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 179/364] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:53 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:54.798333 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:54.798932 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:54.805004 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:54.805550 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:54.806040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:54.806685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:54.807124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:54.807528 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:54.943230 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:54.951082 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a88b51da-19b1-413e-9fd2-9c598d8464fa tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1521 microversion: 2.1 time: 0.156524 Oct 05 14:25:54.951652 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 186/365] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:54 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:55.975959 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:55.976541 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:55.982910 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:55.983402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:55.983830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:55.984511 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:55.984961 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:55.985380 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:56.124925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:56.133293 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-10116e4d-b0fd-4263-ac3e-8969f8f3913b tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1501 microversion: 2.1 time: 0.161834 Oct 05 14:25:56.133924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 180/366] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:55 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1501 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:25:56.155601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:25:56.156214 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:25:56.162646 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:56.163133 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:56.163586 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:56.164253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:25:56.164703 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:25:56.165124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:25:56.303981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:25:56.312195 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9bf5ffe0-a4b1-449b-9947-9a8ed607b7de tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1" status: 200 len: 1515 microversion: 2.1 time: 0.160498 Oct 05 14:25:56.312762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 187/367] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:25:56 2022] GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1 => generated 1515 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:27:59.016255 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=17 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:27:59.016991 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=53 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:27:59.018724 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:27:59.018884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:27:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:27:59.020497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:27:59.020648 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:27:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:27:59.021697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:27:59.021838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:27:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:27:59.026382 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:27:59.026879 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:27:59.038546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:27:59.039035 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:27:59.039460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:27:59.040139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:27:59.040582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:27:59.040997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:27:59.066430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Going to try to terminate instance {{(pid=120605) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:27:59.146804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fbb090a2-e0f7-49ae-9653-5b78101ce985 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 204 len: 0 microversion: 2.1 time: 0.131664 Oct 05 14:27:59.147399 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 181/368] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:27:59 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:27:59.153313 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=27 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:27:59.154121 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=53 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:27:59.155161 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:27:59.155310 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:27:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:27:59.156499 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:27:59.156636 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:27:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:27:59.157696 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:27:59.157815 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:27:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:27:59.163019 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:27:59.163622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:27:59.173665 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:27:59.174362 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:27:59.174792 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:27:59.175530 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:27:59.175976 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:27:59.176388 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:27:59.306794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:27:59.314582 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2e7a0079-8732-4cdd-897c-75af60da74bb tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1679 microversion: 2.1 time: 0.162855 Oct 05 14:27:59.315151 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 188/369] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:27:59 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1679 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:28:00.340778 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:00.341494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:00.347709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:00.348197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:00.348981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:00.349693 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:00.350162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:00.350573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:00.484736 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:00.492992 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-29ecbee8-44c6-4f64-8cb1-3e6cfd1e7d18 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1679 microversion: 2.1 time: 0.156528 Oct 05 14:28:00.493601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 182/370] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:00 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1679 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:01.340419 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-2355da95-d7ae-4616-aff9-204a72ad6545 req-c5566c9f-e6f2-47f1-89da-41ff7ab0b263 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "41b4c31d-52e5-4475-94d1-d54892780716", "name": "network-vif-deleted", "tag": "42cca083-7f47-4f75-a81b-db9d9cef539c"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:28:01.352664 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-2355da95-d7ae-4616-aff9-204a72ad6545 req-c5566c9f-e6f2-47f1-89da-41ff7ab0b263 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.353105 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-2355da95-d7ae-4616-aff9-204a72ad6545 req-c5566c9f-e6f2-47f1-89da-41ff7ab0b263 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.353757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-2355da95-d7ae-4616-aff9-204a72ad6545 req-c5566c9f-e6f2-47f1-89da-41ff7ab0b263 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:01.366675 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-2355da95-d7ae-4616-aff9-204a72ad6545 req-c5566c9f-e6f2-47f1-89da-41ff7ab0b263 service nova] Creating event network-vif-deleted:42cca083-7f47-4f75-a81b-db9d9cef539c for instance 41b4c31d-52e5-4475-94d1-d54892780716 on hv-cinder-83587 Oct 05 14:28:01.372581 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-2355da95-d7ae-4616-aff9-204a72ad6545 req-c5566c9f-e6f2-47f1-89da-41ff7ab0b263 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102079 Oct 05 14:28:01.373164 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 189/371] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:28:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:28:01.520536 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:01.524788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:01.531699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.532188 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.533303 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:01.534056 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.534516 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.535210 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:01.579642 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.579642 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.579642 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:01.579642 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lazy-loading 'fault' on Instance uuid 41b4c31d-52e5-4475-94d1-d54892780716 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:28:01.620209 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:01.629922 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3cd4a3bb-bfd0-491d-8b7d-ad768a8397bf tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 200 len: 1331 microversion: 2.1 time: 0.114183 Oct 05 14:28:01.630604 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 183/372] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:01 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 1331 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:01.810515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:01.811431 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:01.817914 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.818538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.818948 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:01.819631 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.820067 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.820492 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:01.850426 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:28:01.948400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2007d1e2-a333-4c9f-a59e-92667e9abbf0 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "DELETE /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 204 len: 0 microversion: 2.20 time: 0.142230 Oct 05 14:28:01.949002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 190/373] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 5 14:28:01 2022] DELETE /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 05 14:28:01.958364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:01.959354 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:01.965548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.966067 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.966494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:01.967195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:01.967638 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:01.968053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:02.103359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:02.111668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:02.113136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:02.114326 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-800716a3-ad8e-4109-bb0c-41ca141fe086 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1725 microversion: 2.20 time: 0.160155 Oct 05 14:28:02.114873 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 184/374] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:28:01 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:28:02.653227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:02.653937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:02.660218 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:02.660699 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:02.661154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:02.661882 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:02.662327 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:02.662739 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:02.676782 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:28:02.678478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:28:02.680215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:28:02.681202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:28:02.682776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:28:02.684130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:28:02.684130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-19c35d05-1f39-4407-bd71-378ff2bfa988 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.031054 Oct 05 14:28:02.684130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 191/375] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:02 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:28:02.689057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c80eb446-608f-4818-af1a-09e4905e3166 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '.version_select at 0x7f77c98dc280>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:02.716241 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c80eb446-608f-4818-af1a-09e4905e3166 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-637087651" status: 202 len: 0 microversion: 2.1 time: 0.030981 Oct 05 14:28:02.717271 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 185/376] 10.222.0.23 () {58 vars in 1218 bytes} [Wed Oct 5 14:28:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-637087651 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 14:28:03.142213 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:03.142901 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:03.149212 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:03.149805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:03.150302 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:03.151042 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:03.151702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:03.152343 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:03.376041 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d8e805a3-bdbd-4088-88e3-2a494a1ac9ad tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Action: 'create', calling method: .version_select at 0x7f77ca5d7b80>, body: {"keypair": {"name": "tempest-keypair-1333305060", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOWWwsxh/+WocgYqoPkg3GBkmN+Ng4QVR6i56WKr7tiMMea/LjRMXyPKLhKO1YN7UjRFMEDZ81oBQsAIKWLFIc0lHVNkhmeSxEVXA+2zijOvvghzomqOdzrDdH6onYsEDA=="}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:28:03.395372 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-d8e805a3-bdbd-4088-88e3-2a494a1ac9ad tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:28:03.399366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-d8e805a3-bdbd-4088-88e3-2a494a1ac9ad tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for user 2e988e0435464c1d8aa4842fab56dfa4 and project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:28:03.422030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d8e805a3-bdbd-4088-88e3-2a494a1ac9ad tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.049999 Oct 05 14:28:03.422766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 186/377] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:28:03 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:28:03.430629 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:03.439971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:03.441684 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:03.443129 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-503fd262-49db-427f-ac50-91ea4bb93cdf tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1725 microversion: 2.20 time: 0.304836 Oct 05 14:28:03.444199 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 192/378] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:28:03 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1725 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 05 14:28:03.876941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-59d266b4-2044-4ff2-8db5-50c8d67e2170 req-7ab478d1-fbaa-4e84-ba9a-0578100213f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "172fdb3d-4bc0-4955-afed-d2cfe04ce532", "name": "network-vif-deleted", "tag": "040d3759-79ba-48f0-bf0a-1092f124ad90"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:28:03.892210 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-59d266b4-2044-4ff2-8db5-50c8d67e2170 req-7ab478d1-fbaa-4e84-ba9a-0578100213f5 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:03.892769 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-59d266b4-2044-4ff2-8db5-50c8d67e2170 req-7ab478d1-fbaa-4e84-ba9a-0578100213f5 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:03.893194 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-59d266b4-2044-4ff2-8db5-50c8d67e2170 req-7ab478d1-fbaa-4e84-ba9a-0578100213f5 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:03.911634 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-59d266b4-2044-4ff2-8db5-50c8d67e2170 req-7ab478d1-fbaa-4e84-ba9a-0578100213f5 service nova] Creating event network-vif-deleted:040d3759-79ba-48f0-bf0a-1092f124ad90 for instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 on hv-cinder-83587 Oct 05 14:28:03.917444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-59d266b4-2044-4ff2-8db5-50c8d67e2170 req-7ab478d1-fbaa-4e84-ba9a-0578100213f5 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044032 Oct 05 14:28:03.920091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 187/379] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:28:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:28:04.469891 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:04.471103 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:04.477937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:04.478392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:04.479063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:04.480021 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:04.480654 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:04.481257 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:04.528063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:04.528604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:04.529040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:04.529497 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.objects.instance [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lazy-loading 'fault' on Instance uuid 172fdb3d-4bc0-4955-afed-d2cfe04ce532 {{(pid=120606) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:28:04.574439 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:04.585739 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:04.587762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bad6116fbd5c407c9f44084fadd9531e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:04.589114 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b8e94d2c-851b-4ce1-b2d4-e917299f0de2 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 200 len: 1371 microversion: 2.20 time: 0.123150 Oct 05 14:28:04.590357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 193/380] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:28:04 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 1371 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 05 14:28:04.799440 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-143582828", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b6b92f47-a2f9-4f0e-8749-335f8828b651"}], "security_groups": [{"name": "tempest-securitygroup--337706100"}], "key_name": "tempest-keypair-1333305060", "user_data": "}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:28:04.915635 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.network.neutron [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] validate_networks() for [('b6b92f47-a2f9-4f0e-8749-335f8828b651', None, None, None, None, None)] {{(pid=120605) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:28:05.335870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Flavor limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:28:05.336290 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Image limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:28:05.336757 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Flavor pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:28:05.337160 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Image pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:28:05.337578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:28:05.346904 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:05.347555 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:05.347992 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:05.349052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:05.349505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:05.349948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:05.366857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:28:05.370921 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Getting quotas for user 2e988e0435464c1d8aa4842fab56dfa4 and project 45170bafd3c5447cbe6bbb69d7c5f544. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:28:05.383584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Going to run 1 instances... {{(pid=120605) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:28:05.386046 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:28:05.435976 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-dd50874e-8f4c-48ad-a1dc-b42c33c087d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.641425 Oct 05 14:28:05.440691 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 188/381] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:28:04 2022] POST /compute/v2.1/servers => generated 401 bytes in 643 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:28:05.455521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a15eb6f5-5b1a-4f14-9489-f1b372c80dc3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:05.455521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a15eb6f5-5b1a-4f14-9489-f1b372c80dc3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:05.511416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a15eb6f5-5b1a-4f14-9489-f1b372c80dc3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:05.518376 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a15eb6f5-5b1a-4f14-9489-f1b372c80dc3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1241 microversion: 2.1 time: 0.074392 Oct 05 14:28:05.518949 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 194/382] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:05 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1241 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:05.618894 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:05.619806 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:05.630336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:05.630816 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:05.631249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:05.631932 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:05.632379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:05.632792 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:05.646502 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] HTTP exception thrown: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Request method failure captured: Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 HTTP/1.1 Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCIMQVjIopH9J5k4QVCgWqfjnjjnB1gb3lTFqFSV1Bz3OG_QdqDm9-htnPLVrDJYsQHKuvH1VTk6k_hVoMJ7YFtDEnGfOMmCvcvB9_LnJ6fWE_7-LqLbJGn5007TuAXc9zrEDsif-LnJ_z6KZ09Q-u1bdydicPrxSvD0DJZdBktU6s Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Openstack-Nova-Api-Version: 2.20 Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:28:05.648116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_6bd3e20c75e94f3a9c17fcb3029bfdff", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bad6116fbd5c407c9f44084fadd9531e Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:28:05.649843 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '172fdb3d-4bc0-4955-afed-d2cfe04ce532'} Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:28:05.650832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:28:05.652124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:28:05.653387 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Returning 404 to user: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:28:05.653387 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a9ad5457-37ce-48d9-8364-18b8e0eec508 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 404 len: 111 microversion: 2.20 time: 0.036865 Oct 05 14:28:05.653387 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 189/383] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:28:05 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 05 14:28:05.660669 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6dbc0fcc-452d-48c4-9d47-aa7b942e348a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '.version_select at 0x7f77c997f8b0>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:05.686222 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6dbc0fcc-452d-48c4-9d47-aa7b942e348a tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-140775366" status: 204 len: 0 microversion: 2.20 time: 0.031153 Oct 05 14:28:05.686727 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 195/384] 10.222.0.23 () {60 vars in 1259 bytes} [Wed Oct 5 14:28:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-140775366 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 05 14:28:06.541620 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cca3c003-1dab-4865-b929-5d8650562f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:06.542363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-cca3c003-1dab-4865-b929-5d8650562f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:06.597610 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-cca3c003-1dab-4865-b929-5d8650562f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:06.605085 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-cca3c003-1dab-4865-b929-5d8650562f6f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1241 microversion: 2.1 time: 0.068146 Oct 05 14:28:06.605714 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 190/385] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:06 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1241 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:07.629071 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-44f77c8e-53c1-4e07-8088-cd602a1896d3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:07.629699 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-44f77c8e-53c1-4e07-8088-cd602a1896d3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:07.685749 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-44f77c8e-53c1-4e07-8088-cd602a1896d3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:07.693073 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-44f77c8e-53c1-4e07-8088-cd602a1896d3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1241 microversion: 2.1 time: 0.069151 Oct 05 14:28:07.693672 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 196/386] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:07 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1241 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:08.719204 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-874bef29-913f-4fbd-b215-eee5f7a6f3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:08.719968 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-874bef29-913f-4fbd-b215-eee5f7a6f3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:08.779791 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-874bef29-913f-4fbd-b215-eee5f7a6f3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:08.792472 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-874bef29-913f-4fbd-b215-eee5f7a6f3f4 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1241 microversion: 2.1 time: 0.079192 Oct 05 14:28:08.793284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 191/387] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:08 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1241 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:09.819687 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:09.820207 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:09.826645 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:09.827125 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:09.827579 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:09.828363 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:09.828801 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:09.829211 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:09.891958 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:09.899536 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-447566da-8bdb-4cb6-acb6-3bcfa653daab tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1245 microversion: 2.1 time: 0.084482 Oct 05 14:28:09.900129 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 197/388] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:09 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1245 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:10.922247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:10.922854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:10.928981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:10.929588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:10.930048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:10.930738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:10.931185 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:10.931600 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:10.994794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:11.003031 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-838b7b86-8708-4e61-8c84-799f41fc8de1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1299 microversion: 2.1 time: 0.084676 Oct 05 14:28:11.003627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 192/389] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:10 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1299 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:12.027019 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:12.027662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:12.034004 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:12.034497 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:12.034926 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:12.035617 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:12.036068 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:12.036489 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:12.200962 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-17d14a8b-2a70-483f-a241-6ba6adff262b req-04997e9c-5e62-4f25-acef-5feef4e810a8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee", "tag": "95555b20-47f7-4e3b-81ed-c0c576b6f39f"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:28:12.211064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-17d14a8b-2a70-483f-a241-6ba6adff262b req-04997e9c-5e62-4f25-acef-5feef4e810a8 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:12.212475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-17d14a8b-2a70-483f-a241-6ba6adff262b req-04997e9c-5e62-4f25-acef-5feef4e810a8 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:12.213038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-17d14a8b-2a70-483f-a241-6ba6adff262b req-04997e9c-5e62-4f25-acef-5feef4e810a8 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:12.225450 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-17d14a8b-2a70-483f-a241-6ba6adff262b req-04997e9c-5e62-4f25-acef-5feef4e810a8 service nova] Creating event network-changed:95555b20-47f7-4e3b-81ed-c0c576b6f39f for instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee on hv-cinder-83587 Oct 05 14:28:12.230944 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-17d14a8b-2a70-483f-a241-6ba6adff262b req-04997e9c-5e62-4f25-acef-5feef4e810a8 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033722 Oct 05 14:28:12.231776 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 193/390] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:28:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:28:12.242369 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:12.250831 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3a0502c9-95d4-43d1-9fb4-2116eacb5c42 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.227767 Oct 05 14:28:12.252358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 198/391] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:12 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:13.274504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:13.275267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:13.282152 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:13.282658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:13.283107 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:13.283825 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:13.284292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:13.284727 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:13.428531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:13.436805 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4bcb80cf-1a58-4399-b625-90c7e80b906a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.166839 Oct 05 14:28:13.437403 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 194/392] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:13 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:14.463163 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:14.463947 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:14.473955 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:14.474658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:14.475288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:14.476486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:14.476937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:14.477348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:14.614547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:14.623056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-230527d9-e6f9-4af1-959b-173d864eac69 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.165900 Oct 05 14:28:14.623741 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 199/393] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:14 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:15.649173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:15.649784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:15.652398 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:15.658808 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:15.659297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:15.659695 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:15.660459 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:15.661008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:15.661259 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:15.792366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:15.804433 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-122ae0ac-4402-4c18-9a34-b337e8928a7a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.160777 Oct 05 14:28:15.805256 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 195/394] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:15 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:16.832442 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:16.833061 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:16.839646 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:16.840128 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:16.840520 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:16.841223 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:16.841668 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:16.842089 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:16.974839 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:16.992681 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6219a000-f48f-4f46-9fb8-83afdd2a862a tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.155253 Oct 05 14:28:16.993512 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 200/395] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:16 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:18.020314 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:18.021142 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:18.031249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:18.032198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:18.032954 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:18.033862 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:18.034604 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:18.035285 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:18.174186 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:18.182210 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-32c808fe-8c97-4914-aa73-17f07dfb34e9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.168470 Oct 05 14:28:18.183014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 196/396] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:18 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:18.896243 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=4 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:18.964712 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-f8909fc8-5572-4c34-9a95-b8f528944d9a req-88a47c5d-e6a9-400c-9273-42f7ff960ef7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee", "name": "network-vif-plugged", "status": "completed", "tag": "95555b20-47f7-4e3b-81ed-c0c576b6f39f"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:28:18.974793 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-f8909fc8-5572-4c34-9a95-b8f528944d9a req-88a47c5d-e6a9-400c-9273-42f7ff960ef7 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:18.975269 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-f8909fc8-5572-4c34-9a95-b8f528944d9a req-88a47c5d-e6a9-400c-9273-42f7ff960ef7 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:18.975718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-f8909fc8-5572-4c34-9a95-b8f528944d9a req-88a47c5d-e6a9-400c-9273-42f7ff960ef7 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:18.987132 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-f8909fc8-5572-4c34-9a95-b8f528944d9a req-88a47c5d-e6a9-400c-9273-42f7ff960ef7 service nova] Creating event network-vif-plugged:95555b20-47f7-4e3b-81ed-c0c576b6f39f for instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee on hv-cinder-83587 Oct 05 14:28:18.992316 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-f8909fc8-5572-4c34-9a95-b8f528944d9a req-88a47c5d-e6a9-400c-9273-42f7ff960ef7 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100650 Oct 05 14:28:18.992956 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 201/397] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:28:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:28:19.208256 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:19.208862 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:19.214669 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=14 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:19.216552 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:19.217043 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:19.217477 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:19.218206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:19.218651 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:19.219063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:19.357323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:19.365618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d663c44b-0c6e-4c1f-9f54-d91c88d783b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.161234 Oct 05 14:28:19.366213 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 197/398] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:19 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:20.391290 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:20.392215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:20.400762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:20.401400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:20.402027 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:20.402939 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:20.403546 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:20.404107 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:20.540286 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:20.548609 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-887d848f-3a1c-4811-8cc2-3c60cc0da916 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1366 microversion: 2.1 time: 0.163187 Oct 05 14:28:20.549152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 202/399] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:20 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:21.572135 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:21.572716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:21.579248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:21.579714 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:21.580137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:21.580812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:21.581257 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:21.581700 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:21.719473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:28:21.730410 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-df2c572d-d225-4ffa-8399-3aaaaf4a30e1 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1557 microversion: 2.1 time: 0.162139 Oct 05 14:28:21.730954 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 198/400] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:28:21 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1557 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:28:21.752810 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:28:21.753361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:28:21.759721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:21.760213 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:21.760639 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:21.761312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:21.761798 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:21.762279 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:21.849164 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-818381e2-6e65-42bc-8153-d31663d8ad32 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.099961 Oct 05 14:28:21.849778 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 203/401] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 5 14:28:21 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee/os-interface => generated 285 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:28:23.207928 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-b409c316-9bd5-4997-82d7-ff0df6eb9423 req-6536f44a-d0d8-456d-9e7f-89d8f8644f2d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee", "tag": "95555b20-47f7-4e3b-81ed-c0c576b6f39f"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:28:23.217668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b409c316-9bd5-4997-82d7-ff0df6eb9423 req-6536f44a-d0d8-456d-9e7f-89d8f8644f2d service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:28:23.218156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b409c316-9bd5-4997-82d7-ff0df6eb9423 req-6536f44a-d0d8-456d-9e7f-89d8f8644f2d service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:28:23.218581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-b409c316-9bd5-4997-82d7-ff0df6eb9423 req-6536f44a-d0d8-456d-9e7f-89d8f8644f2d service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:28:23.230251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-b409c316-9bd5-4997-82d7-ff0df6eb9423 req-6536f44a-d0d8-456d-9e7f-89d8f8644f2d service nova] Creating event network-changed:95555b20-47f7-4e3b-81ed-c0c576b6f39f for instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee on hv-cinder-83587 Oct 05 14:28:23.235404 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-b409c316-9bd5-4997-82d7-ff0df6eb9423 req-6536f44a-d0d8-456d-9e7f-89d8f8644f2d service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092971 Oct 05 14:28:23.236001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 199/402] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:28:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:29:38.569005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:38.570215 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:38.574394 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:29:38.574882 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:29:38.582725 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:29:38.583208 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:29:38.583635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:29:38.584379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:29:38.584827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:29:38.585240 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:29:38.610323 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-5c305b05-626e-47c8-a9f8-701ce7fdba0c tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee/ips" status: 200 len: 151 microversion: 2.1 time: 0.042865 Oct 05 14:29:38.610867 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 204/403] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:29:38 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee/ips => generated 151 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:30:17.272811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:17.273736 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:17.279609 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f7b99bc5-b086-4572-bfa6-e7ae1e12fcef tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Action: 'create', calling method: .version_select at 0x7f77ca50dee0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1388939984", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIgQx++liXOfwOcj4OS5P9ydZPOSy7Xpm2wreEFfjbQdUkCtIAkX1omDhjg8skt/giNTmZbBKS7Dkn8yxyPwXzo5/iXiHiFiMvvmx9VlW0hFgTTGKpUq55H738df22FVvA=="}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:30:17.299006 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-f7b99bc5-b086-4572-bfa6-e7ae1e12fcef tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Getting quotas for project 126541d90e9c43269898b1249aa6ee9b. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:30:17.303602 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-f7b99bc5-b086-4572-bfa6-e7ae1e12fcef tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Getting quotas for user f109f3d7076543d6bf16eac46a40b3e2 and project 126541d90e9c43269898b1249aa6ee9b. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:30:17.332889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f7b99bc5-b086-4572-bfa6-e7ae1e12fcef tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.056308 Oct 05 14:30:17.332889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 200/404] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:30:17 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:30:17.381427 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2063117099", "imageRef": "7d109ea8-3ebc-4886-8e39-5e6fa0c6db10", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1388939984", "networks": [{"uuid": "d88f93a5-d6f4-42d1-9ef5-a916eead925b"}]}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:30:17.396081 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:17.396721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=4 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:17.445905 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.network.neutron [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] validate_networks() for [('d88f93a5-d6f4-42d1-9ef5-a916eead925b', None, None, None, None, None)] {{(pid=120606) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:30:18.187204 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Flavor limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:30:18.187703 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Image limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:30:18.188318 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Flavor pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:30:18.188830 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Image pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:30:18.189385 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:30:18.202494 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:18.203364 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:18.203822 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:18.204545 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:18.204986 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:18.205397 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:18.222262 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Getting quotas for project 126541d90e9c43269898b1249aa6ee9b. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:30:18.226384 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Getting quotas for user f109f3d7076543d6bf16eac46a40b3e2 and project 126541d90e9c43269898b1249aa6ee9b. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:30:18.237364 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Going to run 1 instances... {{(pid=120606) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:30:18.239656 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='7d109ea8-3ebc-4886-8e39-5e6fa0c6db10',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120606) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:30:18.289921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a89698ae-d9fc-4608-a23f-cb14e5302253 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.912621 Oct 05 14:30:18.290914 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 205/405] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:30:17 2022] POST /compute/v2.1/servers => generated 380 bytes in 915 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:30:18.302340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f21a8180-dadc-44af-9403-91b013f31b54 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:18.303499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f21a8180-dadc-44af-9403-91b013f31b54 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:18.373510 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f21a8180-dadc-44af-9403-91b013f31b54 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:18.394270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f21a8180-dadc-44af-9403-91b013f31b54 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1259 microversion: 2.1 time: 0.084478 Oct 05 14:30:18.394270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 201/406] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:18 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1259 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:19.408428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:19.409731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:19.417876 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:19.418367 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:19.418785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:19.419481 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:19.419922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:19.420327 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:19.493392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:19.502054 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-1eb1aae6-3216-4323-a3bf-3dd364b11131 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1311 microversion: 2.1 time: 0.098227 Oct 05 14:30:19.502956 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 206/407] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:19 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1311 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:20.538694 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:20.543201 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:20.560379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:20.560881 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:20.561323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:20.562042 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:20.562597 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:20.563011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:20.801158 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:20.815649 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-de5d1705-7311-4828-ace5-318c3cf69f98 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.286932 Oct 05 14:30:20.839159 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 202/408] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:20 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:21.849315 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:21.850013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:21.856801 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:21.857283 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:21.857927 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:21.858632 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:21.859089 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:21.859547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:22.188124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:22.197813 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-5e2b657f-1329-47bb-9682-9cdb62b8fc66 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.352915 Oct 05 14:30:22.198767 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 207/409] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:21 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:23.223290 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:23.224242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:23.230440 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:23.231219 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:23.231836 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:23.232709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:23.233540 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:23.233804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:23.440108 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:23.448458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6259ec31-73f0-472c-9e30-858a81f703ac tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.229012 Oct 05 14:30:23.449460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 203/410] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:23 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:24.472591 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:24.473985 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:24.482846 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:24.483757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:24.484305 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:24.485142 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:24.485686 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:24.486326 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:24.665752 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:24.675150 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2ace8cb5-24b1-43b8-a959-882f0fa64e70 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.206607 Oct 05 14:30:24.675723 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 208/411] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:24 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:25.697285 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:25.698243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:25.710578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:25.711164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:25.711685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:25.712453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:25.712969 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:25.717532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:25.942309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:25.951488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-13d2ce38-b3fd-481a-b040-389822aae503 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.257941 Oct 05 14:30:25.952162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 204/412] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:25 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:26.982412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:26.983596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:26.992334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:26.994014 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:26.994554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:26.995357 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:26.995872 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:26.996502 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:27.175845 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:27.184732 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2b1193b4-b599-44d8-a651-91139261b126 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.210292 Oct 05 14:30:27.186082 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 209/413] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:26 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:28.207954 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:28.208846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:28.214847 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:28.215300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:28.215725 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:28.216445 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:28.216887 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:28.217302 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:28.355470 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:28.363607 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-25135855-cdd2-4a83-a967-6975f9cf6d52 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.159585 Oct 05 14:30:28.364181 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 205/414] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:28 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:29.388633 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:29.389332 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:29.392441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:29.393083 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=6 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:29.397615 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:29.398194 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:29.398653 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:29.399352 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:29.399793 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:29.400205 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:29.536037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:29.544074 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3e8ef69b-3d26-4efa-9422-bc2b8a6c4cd3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.159371 Oct 05 14:30:29.548470 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 210/415] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:29 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:30.566546 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:30.567727 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:30.568451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-758c6a05-632d-4c86-8a66-3639cfd0cf42 req-f77fa713-c4a2-49dc-aa36-238f4764be60 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e594ce03-cc8c-409a-b9c1-cb818260ff9a", "tag": "24f4eb3c-2def-48e5-b8fa-61ea31ff7c1a"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:30:30.582461 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=6 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:30.584951 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:30.585881 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-758c6a05-632d-4c86-8a66-3639cfd0cf42 req-f77fa713-c4a2-49dc-aa36-238f4764be60 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:30.586052 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:30.586821 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:30.586960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-758c6a05-632d-4c86-8a66-3639cfd0cf42 req-f77fa713-c4a2-49dc-aa36-238f4764be60 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:30.587597 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:30.587720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-758c6a05-632d-4c86-8a66-3639cfd0cf42 req-f77fa713-c4a2-49dc-aa36-238f4764be60 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:30.588105 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:30.588962 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:30.611562 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-758c6a05-632d-4c86-8a66-3639cfd0cf42 req-f77fa713-c4a2-49dc-aa36-238f4764be60 service nova] Creating event network-changed:24f4eb3c-2def-48e5-b8fa-61ea31ff7c1a for instance e594ce03-cc8c-409a-b9c1-cb818260ff9a on hv-cinder-83587 Oct 05 14:30:30.622863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-758c6a05-632d-4c86-8a66-3639cfd0cf42 req-f77fa713-c4a2-49dc-aa36-238f4764be60 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060912 Oct 05 14:30:30.624347 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 206/416] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:30:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:30:30.858415 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:30.868221 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e44d0ea7-be10-445e-b96d-8a540d3f4d20 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.305897 Oct 05 14:30:30.868802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 211/417] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:30 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:31.893390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:31.894260 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:31.900966 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:31.901453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:31.901983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:31.902706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:31.903118 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:31.903535 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:32.050259 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:32.059276 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f11b1ac7-5161-4f46-85c3-4c21ed44b59a tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.170269 Oct 05 14:30:32.059886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 207/418] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:31 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:33.082195 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:33.082899 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:33.097902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:33.097902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:33.097902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:33.097902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:33.098318 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:33.098318 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:33.239486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:33.251745 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-39d3e213-26b9-4877-8d2d-1722efaa1cf5 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.173407 Oct 05 14:30:33.252249 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 212/419] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:33 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:34.274693 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:34.275918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:34.284512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:34.285192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:34.285619 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:34.286395 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:34.286841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:34.287253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:34.427844 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:34.437865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c7f15c95-3b23-4d71-8110-597153d8eac4 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.167234 Oct 05 14:30:34.438591 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 208/420] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:34 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:35.460620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:35.461857 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:35.468308 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:35.468799 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:35.469239 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:35.469981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:35.470418 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:35.470834 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:35.619767 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:35.629940 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c60c004c-5077-444a-95e6-68a678f36adf tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.173224 Oct 05 14:30:35.631908 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 213/421] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:35 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:36.656827 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:36.657449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:36.664194 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:36.664873 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:36.665539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:36.666440 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:36.666880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:36.667604 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:36.804530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:36.812622 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d40ebbee-1629-46fc-bd1c-f5067bd29b8f tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.160571 Oct 05 14:30:36.813259 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 209/422] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:36 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:37.835918 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:37.836525 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:37.842836 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:37.843519 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:37.844122 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:37.844896 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:37.845336 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:37.845808 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:37.851203 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-633e0dca-fbeb-4306-bef3-f98ae01b4e54 req-0a632d5a-69cc-42b8-bdc3-24e387325dc0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e594ce03-cc8c-409a-b9c1-cb818260ff9a", "name": "network-vif-plugged", "status": "completed", "tag": "24f4eb3c-2def-48e5-b8fa-61ea31ff7c1a"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:30:37.871308 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-633e0dca-fbeb-4306-bef3-f98ae01b4e54 req-0a632d5a-69cc-42b8-bdc3-24e387325dc0 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:37.872095 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-633e0dca-fbeb-4306-bef3-f98ae01b4e54 req-0a632d5a-69cc-42b8-bdc3-24e387325dc0 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:37.872709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-633e0dca-fbeb-4306-bef3-f98ae01b4e54 req-0a632d5a-69cc-42b8-bdc3-24e387325dc0 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:37.885008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-633e0dca-fbeb-4306-bef3-f98ae01b4e54 req-0a632d5a-69cc-42b8-bdc3-24e387325dc0 service nova] Creating event network-vif-plugged:24f4eb3c-2def-48e5-b8fa-61ea31ff7c1a for instance e594ce03-cc8c-409a-b9c1-cb818260ff9a on hv-cinder-83587 Oct 05 14:30:37.892023 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-633e0dca-fbeb-4306-bef3-f98ae01b4e54 req-0a632d5a-69cc-42b8-bdc3-24e387325dc0 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044521 Oct 05 14:30:37.893193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 210/423] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:30:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:30:37.987990 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:37.996658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9a399cbf-82d9-4c2b-86e5-2914fb856d7c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.165126 Oct 05 14:30:37.997221 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 214/424] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:37 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:39.025393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:39.026593 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:39.035086 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:39.035565 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:39.035991 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:39.036670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:39.037111 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:39.037513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:39.171599 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:39.179926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d4d6827c-7b9f-4c75-ac62-722d9ef69225 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1359 microversion: 2.1 time: 0.161655 Oct 05 14:30:39.180493 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 211/425] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:39 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1359 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:40.214314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:40.215285 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:40.224214 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.224691 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.225112 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.225821 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.226259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.226667 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.364077 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:40.372191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-94142ca4-a644-4b58-b2eb-55fe19aa871d tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1549 microversion: 2.1 time: 0.164335 Oct 05 14:30:40.372769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 215/426] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:40 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1549 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:40.395097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:40.395845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:40.402557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.403040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.403467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.404137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.404582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.405002 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.540093 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:40.548155 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-04a7c94d-b73d-4fc0-ac58-46c4dfcb7919 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1549 microversion: 2.1 time: 0.157019 Oct 05 14:30:40.548731 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 212/427] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:40 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1549 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:30:40.570189 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:40.575623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:40.576460 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Searching by: {'deleted': False, 'project_id': '126541d90e9c43269898b1249aa6ee9b'} {{(pid=120606) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 05 14:30:40.583546 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.584086 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.584569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.585293 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.585762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.586181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.679969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.multi_cell_list [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120606) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 05 14:30:40.775517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c0507fe4-3755-489d-ac84-55500f817929 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.209073 Oct 05 14:30:40.776027 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 216/428] 10.222.0.23 () {58 vars in 1125 bytes} [Wed Oct 5 14:30:40 2022] GET /compute/v2.1/servers => generated 343 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:30:40.791476 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:30:40.792603 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:30:40.801044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.801535 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.802011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.802710 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:30:40.803161 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:30:40.803582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:30:40.947801 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:30:40.955493 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9cdb90c4-36a9-4931-9ecb-4189cba9eea0 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1549 microversion: 2.1 time: 0.170796 Oct 05 14:30:40.956060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 213/429] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:30:40 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1549 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:34:06.918986 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:34:06.919795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:34:06.920777 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=70 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:34:06.921589 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:34:06.921779 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:34:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 (ip 10.222.0.23) !!! Oct 05 14:34:06.923517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:34:06.924094 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:34:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 (ip 10.222.0.23) !!! Oct 05 14:34:06.926676 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:34:06.926982 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:34:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 (ip 10.222.0.23) !!! Oct 05 14:34:06.936187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:34:06.936696 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:34:06.951418 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:34:06.951902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:34:06.952323 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:34:06.953038 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:34:06.953725 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:34:06.953995 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:34:06.969279 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] HTTP exception thrown: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Request method failure captured: Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: DELETE /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 HTTP/1.1 Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZCIMQVjIopH9J5k4QVCgWqfjnjjnB1gb3lTFqFSV1Bz3OG_QdqDm9-htnPLVrDJYsQHKuvH1VTk6k_hVoMJ7YFtDEnGfOMmCvcvB9_LnJ6fWE_7-LqLbJGn5007TuAXc9zrEDsif-LnJ_z6KZ09Q-u1bdydicPrxSvD0DJZdBktU6s Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Openstack-Nova-Api-Version: 2.20 Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:34:06.971005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_6bd3e20c75e94f3a9c17fcb3029bfdff", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: bad6116fbd5c407c9f44084fadd9531e Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:06.972731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '172fdb3d-4bc0-4955-afed-d2cfe04ce532'} Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:34:06.973448 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:06.974702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:34:06.975981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Returning 404 to user: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:34:06.975981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c9ffd14a-bff5-4217-9623-c5367a6ff6bb tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "DELETE /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 404 len: 111 microversion: 2.20 time: 0.055082 Oct 05 14:34:06.975981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 217/430] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 5 14:34:06 2022] DELETE /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 05 14:34:06.979271 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:34:06.980010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=64 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:34:06.980990 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:34:06.981158 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:34:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 (ip 10.222.0.23) !!! Oct 05 14:34:06.982279 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:34:06.982396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:34:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 (ip 10.222.0.23) !!! Oct 05 14:34:06.983322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:34:06.983455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:34:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 (ip 10.222.0.23) !!! Oct 05 14:34:06.988116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:34:06.988717 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 172fdb3d-4bc0-4955-afed-d2cfe04ce532] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:34:06.999707 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:34:07.000186 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:34:07.000613 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:34:07.001294 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:34:07.001766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:34:07.002180 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:34:07.017424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] HTTP exception thrown: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Request method failure captured: Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 HTTP/1.1 Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCIMQVjIopH9J5k4QVCgWqfjnjjnB1gb3lTFqFSV1Bz3OG_QdqDm9-htnPLVrDJYsQHKuvH1VTk6k_hVoMJ7YFtDEnGfOMmCvcvB9_LnJ6fWE_7-LqLbJGn5007TuAXc9zrEDsif-LnJ_z6KZ09Q-u1bdydicPrxSvD0DJZdBktU6s Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Openstack-Nova-Api-Version: 2.20 Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:34:07.019100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_6bd3e20c75e94f3a9c17fcb3029bfdff", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bad6116fbd5c407c9f44084fadd9531e Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:07.020391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '172fdb3d-4bc0-4955-afed-d2cfe04ce532'} Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:34:07.020967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. Oct 05 14:34:07.021923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:34:07.022739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Returning 404 to user: Instance 172fdb3d-4bc0-4955-afed-d2cfe04ce532 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:34:07.022739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9032bcff-0fa1-408a-8a79-bf6619ac8c00 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532" status: 404 len: 111 microversion: 2.20 time: 0.043027 Oct 05 14:34:07.022739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 214/431] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:34:06 2022] GET /compute/v2.1/servers/172fdb3d-4bc0-4955-afed-d2cfe04ce532 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 05 14:35:38.233978 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:38.234718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:38.238775 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:35:38.239409 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:35:38.246176 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:38.246650 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:38.247075 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:38.247752 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:38.248192 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:38.248658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:38.276973 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:35:38.360515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2b7769a0-91ff-44c5-aed7-2a745828f3d2 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 204 len: 0 microversion: 2.1 time: 0.127943 Oct 05 14:35:38.361188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 218/432] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:35:38 2022] DELETE /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:35:38.368093 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:38.369336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:38.376636 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:35:38.377248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:35:38.385574 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:38.386109 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:38.386534 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:38.387218 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:38.387659 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:38.388068 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:38.576777 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:35:38.584692 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2cc0de80-3e34-443f-89cc-ebd0c94d8a24 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1682 microversion: 2.1 time: 0.218531 Oct 05 14:35:38.585266 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 215/433] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:35:38 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1682 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:35:39.609760 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:35:39.610475 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:35:39.617940 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:39.617940 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:39.617940 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:39.618597 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:39.619044 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:39.619494 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:39.909922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:35:39.917663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d85ab157-e8a6-41dc-bfac-d52788486815 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1682 microversion: 2.1 time: 0.312360 Oct 05 14:35:39.918266 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 219/434] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:35:39 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1682 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:35:40.398092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-239c9e93-b4c5-47c8-8b1b-cace85dcd20a req-afe335ed-6074-478f-ab23-3a2f505b8f64 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee", "name": "network-vif-deleted", "tag": "95555b20-47f7-4e3b-81ed-c0c576b6f39f"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:35:40.409803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-239c9e93-b4c5-47c8-8b1b-cace85dcd20a req-afe335ed-6074-478f-ab23-3a2f505b8f64 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:40.410349 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-239c9e93-b4c5-47c8-8b1b-cace85dcd20a req-afe335ed-6074-478f-ab23-3a2f505b8f64 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:40.410888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-239c9e93-b4c5-47c8-8b1b-cace85dcd20a req-afe335ed-6074-478f-ab23-3a2f505b8f64 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:40.425245 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-239c9e93-b4c5-47c8-8b1b-cace85dcd20a req-afe335ed-6074-478f-ab23-3a2f505b8f64 service nova] Creating event network-vif-deleted:95555b20-47f7-4e3b-81ed-c0c576b6f39f for instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee on hv-cinder-83587 Oct 05 14:35:40.432466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-239c9e93-b4c5-47c8-8b1b-cace85dcd20a req-afe335ed-6074-478f-ab23-3a2f505b8f64 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.119015 Oct 05 14:35:40.433378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 216/435] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:35:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:35:40.943636 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:35:40.945411 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:35:40.954925 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:40.956063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:40.956607 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:40.957471 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:40.958131 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:40.958943 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:41.000787 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:41.002379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:41.002828 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:41.003312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.objects.instance [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lazy-loading 'fault' on Instance uuid c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee {{(pid=120606) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:35:41.061473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e988e0435464c1d8aa4842fab56dfa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45170bafd3c5447cbe6bbb69d7c5f544', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:35:41.071925 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-508135e6-e3b1-4423-b8cd-c62dfb160f87 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 200 len: 1331 microversion: 2.1 time: 0.134778 Oct 05 14:35:41.073520 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 220/436] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:35:40 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 1331 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:35:42.102272 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:35:42.103301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:35:42.111548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:42.112013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:42.112446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:42.113141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:35:42.113592 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:35:42.114084 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:35:42.128618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee HTTP/1.1 Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:35:42.130269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:35:42.131647 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': 'c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee'} Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:35:42.132353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:35:42.133738 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:35:42.134919 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:35:42.134919 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d1ba064c-a0b8-4f49-adb6-d046d243b8af tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 404 len: 111 microversion: 2.1 time: 0.037160 Oct 05 14:35:42.134919 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 217/437] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:35:42 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:35:42.141047 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4d539983-3858-449e-9cf7-d19fc646ec0e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '.version_select at 0x7f77c9c27430>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:35:42.166686 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4d539983-3858-449e-9cf7-d19fc646ec0e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1333305060" status: 202 len: 0 microversion: 2.1 time: 0.029485 Oct 05 14:35:42.167349 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 221/438] 10.222.0.23 () {58 vars in 1221 bytes} [Wed Oct 5 14:35:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1333305060 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 14:40:07.323574 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:40:07.324122 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:40:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd (ip 10.222.0.23) !!! Oct 05 14:40:07.324967 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:40:07.325075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:40:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd (ip 10.222.0.23) !!! Oct 05 14:40:07.326235 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:40:07.326366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:40:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd (ip 10.222.0.23) !!! Oct 05 14:40:07.327344 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=6 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:07.328193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=10 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:07.334856 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:07.335355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:07.347716 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:07.348193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:07.348621 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:07.349309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:07.349778 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:07.350199 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:07.364965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] HTTP exception thrown: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Request method failure captured: Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd HTTP/1.1 Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCIMQVjIopH9J5k4QVCgWqfjnjjnB1gb3lTFqFSV1Bz3OG_QdqDm9-htnPLVrDJYsQHKuvH1VTk6k_hVoMJ7YFtDEnGfOMmCvcvB9_LnJ6fWE_7-LqLbJGn5007TuAXc9zrEDsif-LnJ_z6KZ09Q-u1bdydicPrxSvD0DJZdBktU6s Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Openstack-Nova-Api-Version: 2.20 Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:40:07.366546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_6bd3e20c75e94f3a9c17fcb3029bfdff", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bad6116fbd5c407c9f44084fadd9531e Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.367920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '13f6cef5-9827-4106-afdc-e572e7f742dd'} Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:40:07.368697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.369871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.371150 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Returning 404 to user: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:40:07.371150 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-994c14b9-c610-4cf8-a889-370ea2b68a02 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "DELETE /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 404 len: 111 microversion: 2.20 time: 0.046285 Oct 05 14:40:07.371150 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 218/439] 10.222.0.23 () {60 vars in 1280 bytes} [Wed Oct 5 14:40:07 2022] DELETE /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 05 14:40:07.374187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:40:07.374710 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:40:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd (ip 10.222.0.23) !!! Oct 05 14:40:07.375564 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:40:07.375680 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:40:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd (ip 10.222.0.23) !!! Oct 05 14:40:07.376608 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 14:40:07.376735 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:40:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd (ip 10.222.0.23) !!! Oct 05 14:40:07.377808 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:07.378556 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:07.384059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:07.384659 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] [instance: 13f6cef5-9827-4106-afdc-e572e7f742dd] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:07.399326 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:07.400040 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:07.400438 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:07.401130 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:07.401576 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:07.402023 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:07.417593 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] HTTP exception thrown: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Request method failure captured: Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd HTTP/1.1 Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZCIMQVjIopH9J5k4QVCgWqfjnjjnB1gb3lTFqFSV1Bz3OG_QdqDm9-htnPLVrDJYsQHKuvH1VTk6k_hVoMJ7YFtDEnGfOMmCvcvB9_LnJ6fWE_7-LqLbJGn5007TuAXc9zrEDsif-LnJ_z6KZ09Q-u1bdydicPrxSvD0DJZdBktU6s Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Openstack-Nova-Api-Version: 2.20 Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:40:07.419154 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_6bd3e20c75e94f3a9c17fcb3029bfdff", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/6bd3e20c75e94f3a9c17fcb3029bfdff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 6bd3e20c75e94f3a9c17fcb3029bfdff Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-526694322 Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: bad6116fbd5c407c9f44084fadd9531e Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-AttachVolumeShelveTestJSON-526694322-project Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.420590 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '13f6cef5-9827-4106-afdc-e572e7f742dd'} Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:40:07.421277 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. Oct 05 14:40:07.422392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:40:07.424175 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] Returning 404 to user: Instance 13f6cef5-9827-4106-afdc-e572e7f742dd could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:40:07.424175 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-70e9d515-4c44-4b62-a82a-4aec92928e25 tempest-AttachVolumeShelveTestJSON-526694322 tempest-AttachVolumeShelveTestJSON-526694322-project] 10.222.0.23 "GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd" status: 404 len: 111 microversion: 2.20 time: 0.047957 Oct 05 14:40:07.426424 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 222/440] 10.222.0.23 () {60 vars in 1277 bytes} [Wed Oct 5 14:40:07 2022] GET /compute/v2.1/servers/13f6cef5-9827-4106-afdc-e572e7f742dd => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 05 14:40:16.552855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-88ffc944-8fe1-4276-bd4b-1ab51fec09d4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'create', calling method: .version_select at 0x7f77cac94ca0>, body: {"keypair": {"name": "tempest-keypair-1239015976", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIUKRZ5hcWpa4cpdXNH4gN+r8kVLdi5plionPVcN/DaSbL0s54fKhy73dK+0bQi03/BJTSkklF2sGt+2wLrTvXDH8JG1Mqxcewk8+bgBcnzwkjiZ7vLJuldRdxFT+8GFZg=="}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:40:16.570416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-88ffc944-8fe1-4276-bd4b-1ab51fec09d4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for project ff81348c32db4e40a8e331317d14e3d2. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:40:16.574698 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-88ffc944-8fe1-4276-bd4b-1ab51fec09d4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for user 7de0d01f891d4d73bc44810aacbca477 and project ff81348c32db4e40a8e331317d14e3d2. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:40:16.602015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-88ffc944-8fe1-4276-bd4b-1ab51fec09d4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.173823 Oct 05 14:40:16.602605 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 219/441] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:40:16 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:40:17.850206 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1926966156", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5c24f205-c58b-44e6-8aed-44447bb6425d"}], "security_groups": [{"name": "tempest-securitygroup--1995200179"}], "key_name": "tempest-keypair-1239015976", "user_data": "}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:40:17.858752 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:17.859445 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:17.962059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.network.neutron [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] validate_networks() for [('5c24f205-c58b-44e6-8aed-44447bb6425d', None, None, None, None, None)] {{(pid=120606) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:40:18.440795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Flavor limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:40:18.441341 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Image limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:40:18.441844 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Flavor pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:40:18.442232 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Image pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:40:18.442643 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:40:18.451401 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:18.451879 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:18.452309 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:18.453032 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:18.453481 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:18.453965 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:18.481152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for project ff81348c32db4e40a8e331317d14e3d2. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:40:18.485402 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for user 7de0d01f891d4d73bc44810aacbca477 and project ff81348c32db4e40a8e331317d14e3d2. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:40:18.497056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Going to run 1 instances... {{(pid=120606) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:40:18.499441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120606) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:40:18.550166 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-1e32225c-b9a5-4c0c-8f63-8b9b8cb581ec tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.704005 Oct 05 14:40:18.557487 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 223/442] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:40:17 2022] POST /compute/v2.1/servers => generated 402 bytes in 706 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:40:18.560139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:18.564847 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-920e4f28-f28f-4cd9-8e50-adf8fa7b95e3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:18.565606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-920e4f28-f28f-4cd9-8e50-adf8fa7b95e3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:18.639591 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-920e4f28-f28f-4cd9-8e50-adf8fa7b95e3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:18.649826 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-920e4f28-f28f-4cd9-8e50-adf8fa7b95e3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1238 microversion: 2.1 time: 0.091049 Oct 05 14:40:18.651379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 220/443] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:18 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1238 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:19.673056 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:19.673810 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:19.681126 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:19.681652 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:19.682113 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:19.683064 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:19.683516 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:19.683931 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:19.750102 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:19.758346 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-73ff6763-3c19-4e30-9fd6-3d874bbb600c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1290 microversion: 2.1 time: 0.089487 Oct 05 14:40:19.758945 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 224/444] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:19 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1290 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:20.782491 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:20.783209 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:20.789520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:20.790059 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:20.790563 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:20.791480 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:20.791935 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:20.792361 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:20.933278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:20.943923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5d7e25de-d9d4-4411-8eb7-cc79244c5d79 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1364 microversion: 2.1 time: 0.165357 Oct 05 14:40:20.954228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 221/445] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:20 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1364 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:21.717796 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-3b8831a2-294d-456f-b588-3859bdf1b913 req-3a3585d6-dd41-403b-afdd-ac9cb14364d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b0af790-f4eb-48b1-b03e-06f2cb5668c7", "tag": "48ab4437-5a55-48e1-ade3-623e768c36db"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:40:21.728227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-3b8831a2-294d-456f-b588-3859bdf1b913 req-3a3585d6-dd41-403b-afdd-ac9cb14364d2 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:21.728678 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-3b8831a2-294d-456f-b588-3859bdf1b913 req-3a3585d6-dd41-403b-afdd-ac9cb14364d2 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:21.729301 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-3b8831a2-294d-456f-b588-3859bdf1b913 req-3a3585d6-dd41-403b-afdd-ac9cb14364d2 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:21.742954 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-3b8831a2-294d-456f-b588-3859bdf1b913 req-3a3585d6-dd41-403b-afdd-ac9cb14364d2 service nova] Creating event network-changed:48ab4437-5a55-48e1-ade3-623e768c36db for instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 on hv-cinder-83587 Oct 05 14:40:21.749083 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-3b8831a2-294d-456f-b588-3859bdf1b913 req-3a3585d6-dd41-403b-afdd-ac9cb14364d2 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035006 Oct 05 14:40:21.749901 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 225/446] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:40:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:40:21.968324 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:21.968933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:21.975087 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:21.975562 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:21.975984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:21.976697 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:21.977522 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:21.977965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:22.107770 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:22.116138 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-40db6193-7f7e-45db-a9f1-66ada3f75e2f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1364 microversion: 2.1 time: 0.152527 Oct 05 14:40:22.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 222/447] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:21 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1364 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:23.139808 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:23.140508 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:23.146987 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:23.147482 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:23.147906 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:23.148598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:23.149045 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:23.149462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:23.289977 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:23.297764 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-010cbac7-4eb3-4e7a-bf5a-b93f284fbdc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1364 microversion: 2.1 time: 0.162420 Oct 05 14:40:23.298301 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 226/448] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:23 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1364 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:24.320548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:24.321190 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:24.327390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:24.327937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:24.328453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:24.329109 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:24.329573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:24.330004 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:24.477788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:24.485490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9d6cce46-deb7-4954-b420-2c5df5252ec5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1364 microversion: 2.1 time: 0.168906 Oct 05 14:40:24.486129 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 223/449] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:24 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1364 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:25.507994 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:25.509167 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:25.516087 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:25.516555 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:25.517044 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:25.517813 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:25.518260 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:25.518673 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:25.665361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:25.674394 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b402adcf-979a-428e-927f-975e759aeedf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1364 microversion: 2.1 time: 0.170358 Oct 05 14:40:25.675307 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 227/450] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:25 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1364 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:26.700393 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:26.701756 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:26.734197 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:26.734745 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:26.735526 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:26.736694 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:26.737330 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:26.737957 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:26.870484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:26.873195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-8db78dbd-1661-4f82-8534-013a2587e66c req-99935649-f38a-4649-a17d-b0fd62671539 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b0af790-f4eb-48b1-b03e-06f2cb5668c7", "name": "network-vif-plugged", "status": "completed", "tag": "48ab4437-5a55-48e1-ade3-623e768c36db"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:40:26.878404 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-d0d28ef0-ca3b-46a5-b988-4e59483ed577 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1364 microversion: 2.1 time: 0.184908 Oct 05 14:40:26.879326 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 228/451] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:26 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1364 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:26.882921 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-8db78dbd-1661-4f82-8534-013a2587e66c req-99935649-f38a-4649-a17d-b0fd62671539 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:26.883389 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-8db78dbd-1661-4f82-8534-013a2587e66c req-99935649-f38a-4649-a17d-b0fd62671539 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:26.883809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-8db78dbd-1661-4f82-8534-013a2587e66c req-99935649-f38a-4649-a17d-b0fd62671539 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:26.894980 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-8db78dbd-1661-4f82-8534-013a2587e66c req-99935649-f38a-4649-a17d-b0fd62671539 service nova] Creating event network-vif-plugged:48ab4437-5a55-48e1-ade3-623e768c36db for instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 on hv-cinder-83587 Oct 05 14:40:26.900947 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-8db78dbd-1661-4f82-8534-013a2587e66c req-99935649-f38a-4649-a17d-b0fd62671539 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.211811 Oct 05 14:40:26.901515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 224/452] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:40:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:40:27.905303 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:27.906336 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:27.916356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:27.917153 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:27.918021 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:27.919145 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:27.919877 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:27.920563 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:28.054833 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:28.062922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0dbc17a8-63cf-411b-b318-f8d1fcd94f90 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1364 microversion: 2.1 time: 0.163962 Oct 05 14:40:28.063489 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 229/453] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:27 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1364 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:29.083491 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=6 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:29.087614 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:29.088306 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:29.095251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:29.095739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:29.096212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:29.096892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:29.097351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:29.097799 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:29.229008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:40:29.237188 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e19fa037-86db-4061-b3f0-9ca7daea579c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1550 microversion: 2.1 time: 0.154744 Oct 05 14:40:29.237785 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 225/454] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:40:29 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1550 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:40:29.259661 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:40:29.260263 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:40:29.262321 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:29.262978 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=6 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:29.268723 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:29.269201 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:29.269643 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:29.270334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:29.270775 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:29.271190 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:29.351144 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6668f6e8-bd0b-4011-962c-1b9be6922f7f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.095311 Oct 05 14:40:29.351783 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 230/455] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 5 14:40:29 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-interface => generated 284 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:40:30.580445 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-a4d5f28f-eaea-403a-8e6d-190ba0505581 req-c5dc7e12-6855-44a6-805e-fde1f5320482 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b0af790-f4eb-48b1-b03e-06f2cb5668c7", "tag": "48ab4437-5a55-48e1-ade3-623e768c36db"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:40:30.590192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-a4d5f28f-eaea-403a-8e6d-190ba0505581 req-c5dc7e12-6855-44a6-805e-fde1f5320482 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:40:30.590842 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-a4d5f28f-eaea-403a-8e6d-190ba0505581 req-c5dc7e12-6855-44a6-805e-fde1f5320482 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:40:30.591325 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-a4d5f28f-eaea-403a-8e6d-190ba0505581 req-c5dc7e12-6855-44a6-805e-fde1f5320482 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:40:30.603903 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-a4d5f28f-eaea-403a-8e6d-190ba0505581 req-c5dc7e12-6855-44a6-805e-fde1f5320482 service nova] Creating event network-changed:48ab4437-5a55-48e1-ade3-623e768c36db for instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 on hv-cinder-83587 Oct 05 14:40:30.609132 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-a4d5f28f-eaea-403a-8e6d-190ba0505581 req-c5dc7e12-6855-44a6-805e-fde1f5320482 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032224 Oct 05 14:40:30.610042 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 226/456] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:40:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:41:35.194026 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:35.194894 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=38 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:35.199398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:41:35.199885 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:41:35.206797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:41:35.207322 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:41:35.207750 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:41:35.208426 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:41:35.208876 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:41:35.209283 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:41:35.233058 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-ad27db32-c993-4cdf-a32f-318475ad8add tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/ips" status: 200 len: 146 microversion: 2.1 time: 0.040203 Oct 05 14:41:35.233663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 231/457] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:41:35 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/ips => generated 146 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:41:42.024472 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:42.025232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=28 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:42.029718 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:41:42.030230 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:41:42.037530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:41:42.038000 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:41:42.038427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:41:42.039106 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:41:42.039546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:41:42.039958 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:41:42.054242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee HTTP/1.1 Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:41:42.055957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.057269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': 'c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee'} Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:41:42.058025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.059246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.060359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:41:42.060359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3dc98548-bf9f-4d9d-8009-8d406eec24a9 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 404 len: 111 microversion: 2.1 time: 0.034443 Oct 05 14:41:42.060949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 227/458] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:41:42 2022] DELETE /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:41:42.066166 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:41:42.066809 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:41:42.073102 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:41:42.073613 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:41:42.074073 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:41:42.074755 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:41:42.075204 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:41:42.075622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:41:42.093551 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee HTTP/1.1 Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:41:42.095271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.096758 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': 'c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee'} Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:41:42.097527 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. Oct 05 14:41:42.098655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:41:42.099646 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:41:42.099646 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-893fa8a9-7d14-4cdf-a17e-7f801d6736b3 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee" status: 404 len: 111 microversion: 2.1 time: 0.034410 Oct 05 14:41:42.099646 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 232/459] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:41:42 2022] GET /compute/v2.1/servers/c9f9d90c-981a-4b0f-b4b0-b4d786ccd9ee => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:42:41.513745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:41.514327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:41.517197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:41.517826 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:41.523708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:41.524183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:41.524611 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:41.525473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:41.525951 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:41.526368 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:41.553286 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Going to try to terminate instance {{(pid=120605) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:42:41.641033 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1ae6f1d7-d704-41c6-9ba0-dd92e85fcebe tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "DELETE /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 204 len: 0 microversion: 2.1 time: 0.130861 Oct 05 14:42:41.641792 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 228/460] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:42:41 2022] DELETE /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:42:41.652543 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:41.653179 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:41.656132 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=4 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:41.656759 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=6 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:41.662484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:41.662953 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:41.663369 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:41.664038 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:41.664473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:41.664866 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:41.968927 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:41.977084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-fe11b37e-5e4b-443f-9a33-6eeda233bd25 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1555 microversion: 2.1 time: 0.329359 Oct 05 14:42:41.977674 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 233/461] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:41 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1555 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:43.001684 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:43.002390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:43.009172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:43.009935 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:43.010360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:43.011337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:43.011961 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:43.012517 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:43.211498 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:43.220037 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a255fbbf-3da9-4541-9b66-284a7ca459a9 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1555 microversion: 2.1 time: 0.222949 Oct 05 14:42:43.220648 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 229/462] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:42 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1555 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:44.246964 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:44.247959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:44.258348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:44.259218 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:44.259674 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:44.260367 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:44.260822 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:44.261254 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:44.394959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:44.404007 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-fbdd337a-4257-4b80-ad38-74db8036fd0b tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1555 microversion: 2.1 time: 0.163494 Oct 05 14:42:44.404569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 234/463] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:44 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1555 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:45.430941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:45.432374 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:45.442424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:45.443219 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:45.443918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:45.445038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:45.445941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:45.446689 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:45.574409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:45.582555 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-53cebcee-bd6d-47ef-bfb3-b6738959fc2e tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1555 microversion: 2.1 time: 0.158081 Oct 05 14:42:45.583156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 230/464] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:45 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1555 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:46.609379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:46.609961 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:46.618037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:46.618510 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:46.619012 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:46.619829 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:46.620280 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:46.620690 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:46.752889 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:46.760830 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e0050216-4d0d-4e1f-8607-0c4bccac0271 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1555 microversion: 2.1 time: 0.157796 Oct 05 14:42:46.761404 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 235/465] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:46 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1555 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:47.790084 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:47.791006 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:47.798146 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:47.798606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:47.799063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:47.799745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:47.800188 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:47.800601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:47.931227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:47.942536 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-40bb4f17-8b08-4ce3-9819-e8c7ef6284a3 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1555 microversion: 2.1 time: 0.159055 Oct 05 14:42:47.943266 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 231/466] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:47 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1555 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:48.966717 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:48.967323 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:48.974583 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:48.975226 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:48.975921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:48.976684 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:48.977158 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:48.977604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:49.181650 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:49.189867 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e08999ff-03fc-4294-8f5b-358b741b3133 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1555 microversion: 2.1 time: 0.227432 Oct 05 14:42:49.190453 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 236/467] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:48 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1555 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:49.462988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-92165d85-df77-4176-9b82-6df798807567 req-f2293557-1c24-47ac-9155-ee57a3c9b206 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e594ce03-cc8c-409a-b9c1-cb818260ff9a", "name": "network-vif-deleted", "tag": "24f4eb3c-2def-48e5-b8fa-61ea31ff7c1a"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:42:49.472920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-92165d85-df77-4176-9b82-6df798807567 req-f2293557-1c24-47ac-9155-ee57a3c9b206 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:49.473395 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-92165d85-df77-4176-9b82-6df798807567 req-f2293557-1c24-47ac-9155-ee57a3c9b206 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:49.473854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-92165d85-df77-4176-9b82-6df798807567 req-f2293557-1c24-47ac-9155-ee57a3c9b206 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:49.485659 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-92165d85-df77-4176-9b82-6df798807567 req-f2293557-1c24-47ac-9155-ee57a3c9b206 service nova] Creating event network-vif-deleted:24f4eb3c-2def-48e5-b8fa-61ea31ff7c1a for instance e594ce03-cc8c-409a-b9c1-cb818260ff9a on hv-cinder-83587 Oct 05 14:42:49.490779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-92165d85-df77-4176-9b82-6df798807567 req-f2293557-1c24-47ac-9155-ee57a3c9b206 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089988 Oct 05 14:42:49.491352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 232/468] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:42:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:42:50.215589 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:50.216545 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:50.223076 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:50.223552 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:50.223969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:50.224645 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:50.225088 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:50.225495 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:50.289831 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:50.300701 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0dc8a00f-07d2-42dc-bfc2-4982de1546cb tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1513 microversion: 2.1 time: 0.090701 Oct 05 14:42:50.301755 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 237/469] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:50 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1513 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:51.325282 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:51.325878 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:51.332802 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:51.333290 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:51.333763 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:51.334458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:51.334909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:51.335326 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:51.405602 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:51.414164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9ce5c3bc-461e-408b-9c32-0f529f0c1472 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1513 microversion: 2.1 time: 0.093525 Oct 05 14:42:51.415040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 233/470] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:51 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1513 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:52.436557 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:52.437160 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:52.443557 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:52.444048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:52.444596 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:52.445667 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:52.446734 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:52.447453 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:52.520997 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:52.526530 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-bbb96a05-30b7-477a-b929-aae1eb280cd1 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1513 microversion: 2.1 time: 0.093966 Oct 05 14:42:52.528428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 238/471] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:52 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1513 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:53.549554 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:53.550436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:53.557926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:53.558500 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:53.559084 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:53.559846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:53.560366 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:53.560844 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:53.624640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f109f3d7076543d6bf16eac46a40b3e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '126541d90e9c43269898b1249aa6ee9b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:42:53.633342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fbf32496-469a-4c6d-8c12-9d27467bdfae tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 200 len: 1344 microversion: 2.1 time: 0.087605 Oct 05 14:42:53.633993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 234/472] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:53 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 1344 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:42:54.660703 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:54.661263 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] [instance: e594ce03-cc8c-409a-b9c1-cb818260ff9a] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:42:54.667287 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:54.667761 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:54.668188 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:54.668869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:42:54.669371 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:42:54.669815 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:42:54.683187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] HTTP exception thrown: Instance e594ce03-cc8c-409a-b9c1-cb818260ff9a could not be found. Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Request method failure captured: Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a HTTP/1.1 Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZTpgv1sMroEdU15s2reyEbnJlXOozA_eQmeLDFz_K6wHKER-cKArME6R4pLmVBm3Hn8ugjUNdgaW7zWhwi4izKVxNDS9tGLaK9MlHsXskfKRapt8j47_4ALXipU8hXZ5kbkVeFptF1jY7UVduIgkjNowwr7Mz2lzfAkBlVhHAp4g7g Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 126541d90e9c43269898b1249aa6ee9b Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-TestMinimumBasicScenario-682041076 Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:42:54.685016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_126541d90e9c43269898b1249aa6ee9b", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/126541d90e9c43269898b1249aa6ee9b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/126541d90e9c43269898b1249aa6ee9b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/126541d90e9c43269898b1249aa6ee9b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-TestMinimumBasicScenario-682041076 Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 126541d90e9c43269898b1249aa6ee9b Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-TestMinimumBasicScenario-682041076 Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-TestMinimumBasicScenario-682041076-project Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: f109f3d7076543d6bf16eac46a40b3e2 Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-TestMinimumBasicScenario-682041076-project Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance e594ce03-cc8c-409a-b9c1-cb818260ff9a could not be found. Oct 05 14:42:54.686771 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': 'e594ce03-cc8c-409a-b9c1-cb818260ff9a'} Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e594ce03-cc8c-409a-b9c1-cb818260ff9a could not be found. Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:42:54.687635 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e594ce03-cc8c-409a-b9c1-cb818260ff9a could not be found. Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e594ce03-cc8c-409a-b9c1-cb818260ff9a could not be found. Oct 05 14:42:54.689451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:42:54.691315 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Returning 404 to user: Instance e594ce03-cc8c-409a-b9c1-cb818260ff9a could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:42:54.691315 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-5102d700-352c-40af-8c14-16932ac73cb7 tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a" status: 404 len: 111 microversion: 2.1 time: 0.029943 Oct 05 14:42:54.691315 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 239/473] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:42:54 2022] GET /compute/v2.1/servers/e594ce03-cc8c-409a-b9c1-cb818260ff9a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:42:54.695873 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-895efe5a-33b6-4cd8-a53f-aac9f165c29c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] Calling method '.version_select at 0x7f77c98f33a0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:42:54.718954 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-895efe5a-33b6-4cd8-a53f-aac9f165c29c tempest-TestMinimumBasicScenario-682041076 tempest-TestMinimumBasicScenario-682041076-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1388939984" status: 202 len: 0 microversion: 2.1 time: 0.026890 Oct 05 14:42:54.719785 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 235/474] 10.222.0.23 () {58 vars in 1272 bytes} [Wed Oct 5 14:42:54 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1388939984 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 14:43:07.544639 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3055bee5-36d3-4d9b-84d8-6d34bb35071d"}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:07.549677 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:07.571606 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=14 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:07.576150 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=38 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:07.585494 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:07.586206 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:07.586717 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:07.587589 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:07.588345 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:07.589086 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:07.855875 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR oslo.messaging._drivers.impl_rabbit [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [1328402e-1398-4987-8d09-3d511f33d3b1] AMQP server on 10.222.0.23:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 05 14:43:08.859565 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [1328402e-1398-4987-8d09-3d511f33d3b1] Reconnected to AMQP server on 10.222.0.23:5672 via [amqp] client with port 44080. Oct 05 14:43:10.483764 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-184eeeb5-e6d9-42fe-8dd9-4360d99bdf26 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.944695 Oct 05 14:43:10.491604 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 240/475] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 5 14:43:07 2022] POST /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments => generated 194 bytes in 2953 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:43:14.910007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=46,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:14.910772 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:15.408287 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-52cf1837-aaed-454e-a90b-7ccc3fa128eb tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Action: 'create', calling method: .version_select at 0x7f77c9be4940>, body: {"keypair": {"name": "tempest-TestShelveInstance-1870409331", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKyOaC5rPdK+7pMU9HohAKSQO5E19XI9zCiK5GeT1ag3kLhxnC20p83YgaCaLLcQphaJ+6OwmCScvROU8gXSGCqJzgLSUJnmJasyGvfhTUWVoUH/5DoZpkaCwarSkuRo5A=="}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:15.448277 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-52cf1837-aaed-454e-a90b-7ccc3fa128eb tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Getting quotas for project 31137b9776dd4dbf89dcc7b10ef4e65b. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:43:15.455623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-52cf1837-aaed-454e-a90b-7ccc3fa128eb tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Getting quotas for user 7859215cd50c4a4f84e27788457dfef6 and project 31137b9776dd4dbf89dcc7b10ef4e65b. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:43:15.494298 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-52cf1837-aaed-454e-a90b-7ccc3fa128eb tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.584453 Oct 05 14:43:15.495889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 236/476] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:43:14 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:43:30.918630 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:30.919835 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:30.925037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-672028818", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "networks": [{"uuid": "83ad587d-481a-4572-af34-a5e33a290738"}]}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:31.011439 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.network.neutron [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] validate_networks() for [('83ad587d-481a-4572-af34-a5e33a290738', None, None, None, None, None)] {{(pid=120606) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:43:31.494793 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Flavor limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:43:31.495193 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Image limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:43:31.495873 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Flavor pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:43:31.496315 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Image pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:43:31.496716 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:43:31.505934 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:31.506620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:31.507222 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:31.507939 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:31.508540 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:31.509091 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:31.528049 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Getting quotas for project 49adc8da67bf4529a349a2bd79e87e07. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:43:31.532667 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Getting quotas for user bc27904d964e43438c32876b8a1b202d and project 49adc8da67bf4529a349a2bd79e87e07. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:43:31.543563 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Going to run 1 instances... {{(pid=120606) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:43:31.546074 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120606) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:43:31.594057 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR oslo.messaging._drivers.impl_rabbit [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [68a8f662-aa28-4be5-9713-baccecf9ce08] AMQP server on 10.222.0.23:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 05 14:43:31.765904 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:31.767014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:31.771014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f432d55a-0ef8-40c3-bad7-a72d6eb45518 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:31.779862 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f432d55a-0ef8-40c3-bad7-a72d6eb45518 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:31.780347 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f432d55a-0ef8-40c3-bad7-a72d6eb45518 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:31.780773 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f432d55a-0ef8-40c3-bad7-a72d6eb45518 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:31.842330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f432d55a-0ef8-40c3-bad7-a72d6eb45518 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Going to try to stop instance {{(pid=120605) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 05 14:43:31.894547 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f432d55a-0ef8-40c3-bad7-a72d6eb45518 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/action" status: 202 len: 0 microversion: 2.1 time: 0.129221 Oct 05 14:43:31.898846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 237/477] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 5 14:43:31 2022] POST /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/action => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 14:43:31.904590 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:31.905153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:31.915519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:31.918173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:31.918613 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:31.919326 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:31.919819 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:31.920608 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:32.094725 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:32.107075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1d56290a-9bff-4e06-b49b-892eb64f53cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.206714 Oct 05 14:43:32.107744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 238/478] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:31 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:32.608924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [68a8f662-aa28-4be5-9713-baccecf9ce08] Reconnected to AMQP server on 10.222.0.23:5672 via [amqp] client with port 45040. Oct 05 14:43:32.613826 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a1550559-9ce2-4e43-ac21-69f63edfbd1b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.696643 Oct 05 14:43:32.614836 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 241/479] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:43:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 1698 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:43:32.626318 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-912c7a51-d512-4614-a5db-43a87628f95e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:32.627251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-912c7a51-d512-4614-a5db-43a87628f95e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:32.691932 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-912c7a51-d512-4614-a5db-43a87628f95e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:32.699620 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-912c7a51-d512-4614-a5db-43a87628f95e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1218 microversion: 2.1 time: 0.077512 Oct 05 14:43:32.700267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 239/480] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:32 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1218 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:33.131691 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:33.132439 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:33.146581 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:33.147323 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:33.148090 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:33.148958 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:33.149404 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:33.150012 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:33.388905 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:33.397398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-17b20c82-514f-465a-857a-f2c747573e73 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.270541 Oct 05 14:43:33.398016 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 242/481] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:33 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:33.723492 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:33.724097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:33.730053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:33.730535 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:33.730962 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:33.731639 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:33.732076 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:33.732658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:33.803766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:33.811692 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d1bc50d5-0598-4e56-8f24-d77ee6cf9bb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1270 microversion: 2.1 time: 0.091919 Oct 05 14:43:33.812428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 240/482] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:33 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1270 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:34.421043 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:34.421859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:34.428696 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:34.430253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:34.430776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:34.432462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:34.433102 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:34.433597 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:34.656401 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:34.664871 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4eb7b5d2-8214-4855-9e87-6aa0f33a7ca3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.248327 Oct 05 14:43:34.665442 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 243/483] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:34 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:34.838792 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:34.839832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:34.847770 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:34.848255 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:34.848683 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:34.849384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:34.849857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:34.850276 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:35.038314 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:35.046745 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8ec44bc2-ac19-4603-8164-839c5619d6e0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.214025 Oct 05 14:43:35.047529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 241/484] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:34 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:43:35.690427 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:35.692216 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:35.701177 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:35.701935 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:35.702362 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:35.703080 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:35.703530 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:35.703944 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:35.867007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-375e6b2c-181e-48e3-a859-b96d32ac1795 req-64c1657d-af1c-41cd-aff8-77d15afc0580 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3cb025a5-578e-4cef-9ba2-3bc0d4e17633", "tag": "235589b0-b52d-44dc-9b6d-b670d6128608"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:35.890129 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-375e6b2c-181e-48e3-a859-b96d32ac1795 req-64c1657d-af1c-41cd-aff8-77d15afc0580 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:35.890999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-375e6b2c-181e-48e3-a859-b96d32ac1795 req-64c1657d-af1c-41cd-aff8-77d15afc0580 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:35.892539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-375e6b2c-181e-48e3-a859-b96d32ac1795 req-64c1657d-af1c-41cd-aff8-77d15afc0580 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:35.906047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-375e6b2c-181e-48e3-a859-b96d32ac1795 req-64c1657d-af1c-41cd-aff8-77d15afc0580 service nova] Creating event network-changed:235589b0-b52d-44dc-9b6d-b670d6128608 for instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 on hv-cinder-83587 Oct 05 14:43:35.914907 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-375e6b2c-181e-48e3-a859-b96d32ac1795 req-64c1657d-af1c-41cd-aff8-77d15afc0580 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061478 Oct 05 14:43:35.919474 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 242/485] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:43:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:43:35.949408 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:35.960406 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f4cbb0bf-5f29-49ac-90eb-081408f01fb8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.275864 Oct 05 14:43:35.960980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 244/486] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:35 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:36.072811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:36.074779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:36.083746 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:36.084722 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:36.085321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:36.086739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:36.087514 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:36.088096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:36.260129 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:36.271794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-bd909201-7521-442a-9b91-00f9e301d8e3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.204762 Oct 05 14:43:36.273017 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 243/487] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:36 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:43:36.985965 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:36.986892 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:36.994293 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:36.995093 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:36.995625 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:36.996392 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:36.996894 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:36.997938 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:37.170420 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:37.180228 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2c07655d-d3e3-494d-8e39-2704afbaf560 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.199636 Oct 05 14:43:37.180815 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 245/488] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:36 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:37.297105 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:37.297824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:37.303970 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:37.304649 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:37.305237 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:37.305962 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:37.306572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:37.306993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:37.457139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:37.465003 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-741cb52f-7f9b-46a2-9267-49557f7c7d92 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.173421 Oct 05 14:43:37.466031 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 244/489] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:37 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:38.202964 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:38.204261 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:38.211079 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:38.211658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:38.212619 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:38.213586 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:38.214299 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:38.214939 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:38.394933 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:38.408179 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-9c44edbf-4a36-4ca7-929a-aca490d3d297 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.209194 Oct 05 14:43:38.426059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 246/490] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:38 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:38.488939 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:38.490798 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:38.506632 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:38.508272 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:38.508733 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:38.511381 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:38.512026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:38.513318 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:38.763670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:38.772862 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-553647a0-5c29-49b0-8d13-3316df240852 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.288040 Oct 05 14:43:38.786395 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 245/491] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:38 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:39.442501 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:39.443106 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:39.455550 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:39.456046 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:39.456481 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:39.462545 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:39.463088 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:39.463535 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:39.796446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:39.797061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:39.804382 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:39.804878 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:39.805308 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:39.806054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:39.807720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:39.808585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:39.809390 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:39.820339 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3eaf336f-2b28-43df-8f29-05702d602655 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.385656 Oct 05 14:43:39.821658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 247/492] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:39 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:40.119055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:40.134763 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b9210ab4-6db7-4e22-84b6-92ddb274ba63 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.342153 Oct 05 14:43:40.135353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 246/493] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:39 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:43:40.856471 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:40.857858 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:40.871428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:40.871900 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:40.872500 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:40.873391 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:40.874068 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:40.874848 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:41.158695 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:41.159889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:41.170355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:41.171525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:41.172784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:41.179784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:41.180605 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:41.181262 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:41.248855 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:41.258698 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c46f07f4-d96c-46ab-93e7-2399a1be1fcb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.417620 Oct 05 14:43:41.262153 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 248/494] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:40 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:41.732385 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:41.741228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f1428a93-e2db-4e64-81cb-5752c6dc5307 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.587278 Oct 05 14:43:41.757569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 247/495] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:41 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:42.288920 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:42.289584 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:42.292542 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:42.293207 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=6 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:42.303486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:42.304282 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:42.304880 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:42.305803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:42.306741 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:42.307371 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:42.619038 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:42.629398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8696cdf7-6b5d-4779-af54-d93f413cbd06 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.344781 Oct 05 14:43:42.649833 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 249/496] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:42 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:42.766880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:42.767530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:42.773282 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:42.811764 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:42.812480 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.027s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:42.825119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:42.828219 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:42.833286 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:42.835270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:43.220568 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:43.234111 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-53d5f865-d5e3-454b-8a5e-fb7622e0617c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.471177 Oct 05 14:43:43.250566 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 248/497] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:42 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:43.434418 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-182d2ea4-d797-4904-a370-531ff2f8987f req-80f7de33-96a6-4f30-b0ba-a01aae0e6834 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3cb025a5-578e-4cef-9ba2-3bc0d4e17633", "name": "network-vif-plugged", "status": "completed", "tag": "235589b0-b52d-44dc-9b6d-b670d6128608"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:43.446575 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-182d2ea4-d797-4904-a370-531ff2f8987f req-80f7de33-96a6-4f30-b0ba-a01aae0e6834 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:43.447055 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-182d2ea4-d797-4904-a370-531ff2f8987f req-80f7de33-96a6-4f30-b0ba-a01aae0e6834 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:43.447805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-182d2ea4-d797-4904-a370-531ff2f8987f req-80f7de33-96a6-4f30-b0ba-a01aae0e6834 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:43.470997 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-182d2ea4-d797-4904-a370-531ff2f8987f req-80f7de33-96a6-4f30-b0ba-a01aae0e6834 service nova] Creating event network-vif-plugged:235589b0-b52d-44dc-9b6d-b670d6128608 for instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 on hv-cinder-83587 Oct 05 14:43:43.477559 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-182d2ea4-d797-4904-a370-531ff2f8987f req-80f7de33-96a6-4f30-b0ba-a01aae0e6834 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046980 Oct 05 14:43:43.480595 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 250/498] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:43:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:43:43.661566 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:43.667040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:43.691165 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:43.691744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:43.692179 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:43.713595 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:43.713595 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:43.713595 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:44.259340 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:44.260118 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:44.267227 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:44.267721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:44.268603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:44.268801 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:44.269892 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:44.270313 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:44.318839 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:44.329485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c24bf277-b881-4a6a-afd9-f4f0443ba1e1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.678689 Oct 05 14:43:44.334064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 249/499] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:43 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 684 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:44.444763 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:44.457376 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-cff94efe-39a5-4bd0-ab28-51b44636a72e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.202442 Oct 05 14:43:44.458372 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 251/500] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:44 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:45.355347 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:45.356001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:45.368402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:45.368959 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:45.370913 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:45.371658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:45.372124 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:45.372550 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:45.489309 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:45.490595 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:45.499230 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:45.506025 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:45.506794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:45.508260 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:45.510155 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:45.511210 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:45.728124 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:45.737613 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-279185ee-2e54-4200-aded-06df3f40f6dc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1318 microversion: 2.1 time: 0.251245 Oct 05 14:43:45.738265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 252/501] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:45 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1318 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:43:45.743532 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:45.754478 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0bd65637-75dd-4252-919c-643f6f8c56f8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.405494 Oct 05 14:43:45.757760 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 250/502] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:45 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:46.762901 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:46.763642 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:46.770286 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:46.770797 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:46.771234 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:46.771996 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:46.772398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:46.772872 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:46.783618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:46.785675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:46.797409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:46.798756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:46.799830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:46.801369 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:46.802458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:46.803537 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:46.947335 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:46.955330 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-66de5008-e2a3-42b2-8b7a-56c14c284fee tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1507 microversion: 2.1 time: 0.196246 Oct 05 14:43:46.955919 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 253/503] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:46 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1507 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:47.100178 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:47.114001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a5b8c762-6b13-4c40-9a98-f47ee01b7675 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1725 microversion: 2.1 time: 0.336424 Oct 05 14:43:47.114955 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 251/504] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:46 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1725 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:48.137657 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:48.139189 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:48.148005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:48.148495 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:48.148936 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:48.149672 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:48.150108 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:48.150732 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:48.357116 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:48.367928 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6cf2765d-f8ab-44df-a927-6c0cf52fead6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1702 microversion: 2.1 time: 0.234050 Oct 05 14:43:48.368988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 254/505] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:48 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1702 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:48.390928 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8714a4e3-a2c9-4ef0-9c13-8363f6b54e93 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:48.397769 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8714a4e3-a2c9-4ef0-9c13-8363f6b54e93 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:48.398544 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8714a4e3-a2c9-4ef0-9c13-8363f6b54e93 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:48.399125 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8714a4e3-a2c9-4ef0-9c13-8363f6b54e93 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:48.448091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8714a4e3-a2c9-4ef0-9c13-8363f6b54e93 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Going to try to start instance {{(pid=120605) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 05 14:43:48.518318 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8714a4e3-a2c9-4ef0-9c13-8363f6b54e93 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/action" status: 202 len: 0 microversion: 2.1 time: 0.130756 Oct 05 14:43:48.519224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 252/506] 10.222.0.23 () {60 vars in 1278 bytes} [Wed Oct 5 14:43:48 2022] POST /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/action => generated 0 bytes in 133 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 14:43:48.559355 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:48.560014 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:48.589524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:48.590679 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:48.591465 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:48.592404 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:48.593147 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:48.593807 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:48.914363 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:48.925684 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0425c0ab-e9d9-40fa-9c53-270071b4245c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1711 microversion: 2.1 time: 0.400891 Oct 05 14:43:48.926816 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 255/507] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:48 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1711 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:49.955261 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:49.956744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:49.967226 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:49.968083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:49.968664 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:49.969767 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:49.970708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:49.971297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:49.988771 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:50.299401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:50.309256 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-cad538a5-757f-468a-a945-b13cf123d0ff tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1711 microversion: 2.1 time: 0.361404 Oct 05 14:43:50.310530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 253/508] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:49 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1711 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:43:51.334779 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:51.336864 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:51.348200 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:51.350663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:51.351105 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:51.351855 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:51.352304 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:51.352722 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:51.667347 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:51.676850 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-33c86985-da06-4b29-a136-ea6bd532545e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1711 microversion: 2.1 time: 0.346291 Oct 05 14:43:51.678262 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 256/509] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:51 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1711 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:52.051447 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-be0ccd58-a806-480d-b05e-9882445f25cf tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Action: 'create', calling method: .version_select at 0x7f77c9b96ca0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-648941858", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPcDzsy8PSOjVYqGdIveUerNiuwXG+ydon6zrexcWT9OBn2w4dKL9myfo1IMMBH+DZDCAJ6iJjDj6LojIXwDowSZiTtzoJ5E2Yj1woy3f99zKum30rqmFBsJy/lqXx7uxg=="}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:52.066775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-be0ccd58-a806-480d-b05e-9882445f25cf tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Getting quotas for project 93588cc1d2704a73ae5a76dcc6fc44f1. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:43:52.070674 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-be0ccd58-a806-480d-b05e-9882445f25cf tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Getting quotas for user dea49d304629415895813b6ab7f9c9db and project 93588cc1d2704a73ae5a76dcc6fc44f1. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:43:52.097144 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-be0ccd58-a806-480d-b05e-9882445f25cf tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.233972 Oct 05 14:43:52.098050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 254/510] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:43:51 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:43:52.704643 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:52.705658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:52.713398 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:52.714442 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:52.714828 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:52.716088 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:52.716534 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:52.716980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:52.896607 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:52.908434 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-504f7698-c05d-4e14-9e12-8e2902022110 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1711 microversion: 2.1 time: 0.209299 Oct 05 14:43:52.914502 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 257/511] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:52 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1711 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:53.932618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:53.933221 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:53.939754 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:53.940247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:53.940678 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:53.941372 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:53.941850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:53.942270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:54.074209 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:54.082763 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-862c28c2-5d8c-4451-9609-45d045f9a5bf tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1711 microversion: 2.1 time: 0.154274 Oct 05 14:43:54.083347 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 255/512] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:53 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1711 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:55.109961 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:55.110561 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:55.117425 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:55.118238 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:55.118662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:55.119356 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:55.119795 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:55.120205 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:55.263673 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:55.267862 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=100 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:55.275673 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-5746b8b6-3012-48ff-9588-ec952550fbb0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1711 microversion: 2.1 time: 0.169755 Oct 05 14:43:55.277113 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 258/513] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:55 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1711 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:56.300322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:56.300936 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:56.307206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:56.307695 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:56.308104 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:56.308791 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:56.309245 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:56.309729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:56.446164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:56.454706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2c53e647-5548-4f86-adca-aa9af0001eac tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1711 microversion: 2.1 time: 0.158294 Oct 05 14:43:56.455256 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 256/514] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:56 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1711 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:57.478471 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:57.479275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:57.486714 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:57.487359 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:57.487887 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:57.488722 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:57.489352 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:57.490031 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:57.625826 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:57.634846 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-35aedbf0-3e5b-4220-9526-be999a5dfcf0 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1715 microversion: 2.1 time: 0.160272 Oct 05 14:43:57.635574 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 259/515] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:57 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1715 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:57.974134 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-80e1685a-35b4-42da-a318-3d0c35163667 tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-257714969", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1870409331", "security_groups": [{"name": "tempest-secgroup-smoke-1011708186"}], "networks": [{"uuid": "ff1bd49b-848d-42c6-847e-c83ec9a0dfcf"}], "block_device_mapping_v2": [{"uuid": "f1934bec-d062-498f-8442-e8ee1c6e0271", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:43:58.420694 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:58.421195 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:58.428827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:58.429345 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:58.429792 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:58.430461 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:58.431252 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:58.431801 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:58.459486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:43:58.557096 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-fe39b1a7-0c41-402f-a050-5cce629c8f19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "DELETE /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 204 len: 0 microversion: 2.1 time: 0.139900 Oct 05 14:43:58.558118 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 260/516] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:43:58 2022] DELETE /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 05 14:43:58.567589 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:58.568304 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:58.575049 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:58.575580 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:58.575976 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:58.576711 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:58.577187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:58.577603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:58.734183 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:58.742511 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a962e275-3db5-4848-8d59-bbc2e9b29c6e tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1507 microversion: 2.1 time: 0.179267 Oct 05 14:43:58.743460 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 261/517] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:58 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1507 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:43:59.767048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:43:59.767654 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:43:59.774280 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:59.774761 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:59.775185 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:59.775861 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:43:59.776305 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:43:59.776711 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:43:59.930084 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:43:59.938937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6302283d-cf26-40d3-9def-b7cdee07c014 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1507 microversion: 2.1 time: 0.175873 Oct 05 14:43:59.939617 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 262/518] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:43:59 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1507 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:00.777345 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-29315bae-1c45-45c5-9e23-6b1dbdf808fa req-62d3d2ca-c1bf-4757-bc1c-58fd184f14e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db6f4760-f8d9-4962-bdb0-623a7f51267c", "name": "network-vif-deleted", "tag": "47e256dc-16ba-45a9-a0db-f7e74740b0b0"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:44:00.794812 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-29315bae-1c45-45c5-9e23-6b1dbdf808fa req-62d3d2ca-c1bf-4757-bc1c-58fd184f14e7 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:00.795833 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-29315bae-1c45-45c5-9e23-6b1dbdf808fa req-62d3d2ca-c1bf-4757-bc1c-58fd184f14e7 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:00.796486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-29315bae-1c45-45c5-9e23-6b1dbdf808fa req-62d3d2ca-c1bf-4757-bc1c-58fd184f14e7 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:00.809196 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-29315bae-1c45-45c5-9e23-6b1dbdf808fa req-62d3d2ca-c1bf-4757-bc1c-58fd184f14e7 service nova] Creating event network-vif-deleted:47e256dc-16ba-45a9-a0db-f7e74740b0b0 for instance db6f4760-f8d9-4962-bdb0-623a7f51267c on hv-cinder-83587 Oct 05 14:44:00.821853 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-29315bae-1c45-45c5-9e23-6b1dbdf808fa req-62d3d2ca-c1bf-4757-bc1c-58fd184f14e7 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.401463 Oct 05 14:44:00.824125 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 263/519] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:44:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:44:00.966295 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:00.967269 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:00.971023 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:00.980573 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:00.981570 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:00.982111 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:00.982923 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:00.983334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:00.983748 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:01.021555 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:01.022324 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:01.022768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:01.023198 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.objects.instance [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lazy-loading 'fault' on Instance uuid db6f4760-f8d9-4962-bdb0-623a7f51267c {{(pid=120606) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:44:01.076634 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:01.085532 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-502ecb4c-df32-4ea9-ac88-07f29ed01d19 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 200 len: 1311 microversion: 2.1 time: 0.126533 Oct 05 14:44:01.086139 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 264/520] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:00 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 1311 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:02.077995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:02.111547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:02.112191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: db6f4760-f8d9-4962-bdb0-623a7f51267c] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:02.119454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:02.120312 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:02.121338 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:02.122151 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:02.122859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:02.123524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:02.141493 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] HTTP exception thrown: Instance db6f4760-f8d9-4962-bdb0-623a7f51267c could not be found. Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Request method failure captured: Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c HTTP/1.1 Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZPBXRIbqqvR7yIPcX--GXbnlkKcYupQTbyg76o1yb0Ltgb2hSJoe02DAewefqngwUruE7bIYWa0XPiJ-ATZa8ZH6APhN4fSbKq6eIAUVEbUrTIoi1BwAUvNKbWll2na7NC5AB_iagPuyuYgrLo8S0ikHCdp_hfkzgE-f7Dye1xbLhw Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: ae3d0e9a5de8481fa19a58f7780a4b7a Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-137184395 Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:44:02.143436 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ae3d0e9a5de8481fa19a58f7780a4b7a", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ae3d0e9a5de8481fa19a58f7780a4b7a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ae3d0e9a5de8481fa19a58f7780a4b7a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ae3d0e9a5de8481fa19a58f7780a4b7a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-ServerRescueNegativeTestJSON-137184395 Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: ae3d0e9a5de8481fa19a58f7780a4b7a Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-137184395 Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-ServerRescueNegativeTestJSON-137184395-project Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 6e4a13a6981f4e0898dc22ea8525a1ee Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-ServerRescueNegativeTestJSON-137184395-project Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance db6f4760-f8d9-4962-bdb0-623a7f51267c could not be found. Oct 05 14:44:02.144800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': 'db6f4760-f8d9-4962-bdb0-623a7f51267c'} Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db6f4760-f8d9-4962-bdb0-623a7f51267c could not be found. Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:44:02.145553 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db6f4760-f8d9-4962-bdb0-623a7f51267c could not be found. Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db6f4760-f8d9-4962-bdb0-623a7f51267c could not be found. Oct 05 14:44:02.146781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:02.148915 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Returning 404 to user: Instance db6f4760-f8d9-4962-bdb0-623a7f51267c could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:44:02.149952 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a964f950-6ba1-43b1-9a46-c49a91f428ce tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c" status: 404 len: 111 microversion: 2.1 time: 0.044225 Oct 05 14:44:02.153978 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 265/521] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:02 2022] GET /compute/v2.1/servers/db6f4760-f8d9-4962-bdb0-623a7f51267c => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 05 14:44:02.159021 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:02.159515 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:02.165591 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:02.166164 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:02.166598 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:02.167297 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:02.167773 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:02.168164 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:02.196710 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:44:02.326883 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a0710ec2-7d6e-49e4-b813-8937d0091744 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "DELETE /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1" status: 204 len: 0 microversion: 2.1 time: 0.171341 Oct 05 14:44:02.328525 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 266/522] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:44:02 2022] DELETE /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:44:02.341361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:02.342411 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:02.352293 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:02.353575 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:02.354310 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:02.355731 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:02.356187 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:02.356614 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:02.505380 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:02.518286 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-afa25f1e-b3a3-4bc0-9a29-8b9bb7bdf1da tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1" status: 200 len: 1521 microversion: 2.1 time: 0.183916 Oct 05 14:44:02.518866 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 267/523] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:02 2022] GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:03.542597 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:03.543230 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:03.551210 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:03.551865 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:03.552358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:03.553126 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:03.553666 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:03.554176 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:03.689504 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e4a13a6981f4e0898dc22ea8525a1ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:03.701092 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c6149caa-174e-481e-95c1-f300ccabcd66 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1" status: 200 len: 1521 microversion: 2.1 time: 0.162366 Oct 05 14:44:03.701721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 268/524] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:03 2022] GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1 => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:03.958551 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-5d5c70fc-097f-4530-ba56-bc92c7cbcde8 req-109bd226-f66c-4d33-aecb-cd74e368d609 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "451ebffc-7a95-491f-9c6b-6f508bc584d1", "name": "network-vif-deleted", "tag": "b772ad13-fc5f-48d5-b449-22494f53ecf8"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:44:03.969945 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-5d5c70fc-097f-4530-ba56-bc92c7cbcde8 req-109bd226-f66c-4d33-aecb-cd74e368d609 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:03.970422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-5d5c70fc-097f-4530-ba56-bc92c7cbcde8 req-109bd226-f66c-4d33-aecb-cd74e368d609 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:03.970841 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-5d5c70fc-097f-4530-ba56-bc92c7cbcde8 req-109bd226-f66c-4d33-aecb-cd74e368d609 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:03.985791 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-5d5c70fc-097f-4530-ba56-bc92c7cbcde8 req-109bd226-f66c-4d33-aecb-cd74e368d609 service nova] Creating event network-vif-deleted:b772ad13-fc5f-48d5-b449-22494f53ecf8 for instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 on hv-cinder-83587 Oct 05 14:44:03.991662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-5d5c70fc-097f-4530-ba56-bc92c7cbcde8 req-109bd226-f66c-4d33-aecb-cd74e368d609 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036669 Oct 05 14:44:03.992288 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 269/525] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:44:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:44:04.636756 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-2122197073", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452"}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:44:06.450687 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=16 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:08.009418 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:12.377687 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.network.neutron [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] validate_networks() for None {{(pid=120606) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:44:12.664029 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Flavor limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:44:12.664451 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Image limits 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:44:12.664921 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Flavor pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:44:12.665334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Image pref 0:0:0 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:44:12.665790 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.virt.hardware [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120606) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:44:12.674836 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:12.675355 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:12.675800 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:12.676503 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:12.676946 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:12.677353 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:12.694733 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Getting quotas for project bea771b3ff124fc8b9b9ac57f8ea12b4. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:44:12.698701 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.quota [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Getting quotas for user 005d0f9318534cfc98c37d822e49bdc4 and project bea771b3ff124fc8b9b9ac57f8ea12b4. Resources: {'instances', 'ram', 'cores'} {{(pid=120606) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:44:12.709959 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Going to run 1 instances... {{(pid=120606) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:44:12.712325 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120606) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:44:12.758776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b0ed0943-889e-46c7-8cb0-38bad6a18af0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 8.126688 Oct 05 14:44:12.759444 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 270/526] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:44:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 8128 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:44:12.764584 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:12.765551 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] [instance: 451ebffc-7a95-491f-9c6b-6f508bc584d1] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:12.771904 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:12.772395 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:12.772826 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:12.773511 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:12.774004 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:12.774764 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:12.789012 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] HTTP exception thrown: Instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 could not be found. Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Request method failure captured: Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1 HTTP/1.1 Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZPBXRIbqqvR7yIPcX--GXbnlkKcYupQTbyg76o1yb0Ltgb2hSJoe02DAewefqngwUruE7bIYWa0XPiJ-ATZa8ZH6APhN4fSbKq6eIAUVEbUrTIoi1BwAUvNKbWll2na7NC5AB_iagPuyuYgrLo8S0ikHCdp_hfkzgE-f7Dye1xbLhw Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: ae3d0e9a5de8481fa19a58f7780a4b7a Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-137184395 Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 14:44:12.790753 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ae3d0e9a5de8481fa19a58f7780a4b7a", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ae3d0e9a5de8481fa19a58f7780a4b7a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ae3d0e9a5de8481fa19a58f7780a4b7a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ae3d0e9a5de8481fa19a58f7780a4b7a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-ServerRescueNegativeTestJSON-137184395 Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: ae3d0e9a5de8481fa19a58f7780a4b7a Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-137184395 Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-ServerRescueNegativeTestJSON-137184395-project Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 6e4a13a6981f4e0898dc22ea8525a1ee Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-ServerRescueNegativeTestJSON-137184395-project Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 could not be found. Oct 05 14:44:12.792152 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '451ebffc-7a95-491f-9c6b-6f508bc584d1'} Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 could not be found. Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:44:12.793035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 could not be found. Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 could not be found. Oct 05 14:44:12.794361 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 14:44:12.795454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] Returning 404 to user: Instance 451ebffc-7a95-491f-9c6b-6f508bc584d1 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:44:12.795454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-72dc7af2-a93b-4a8c-a6ac-dcd618c562f8 tempest-ServerRescueNegativeTestJSON-137184395 tempest-ServerRescueNegativeTestJSON-137184395-project] 10.222.0.23 "GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1" status: 404 len: 111 microversion: 2.1 time: 0.031782 Oct 05 14:44:12.795454 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 271/527] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:12 2022] GET /compute/v2.1/servers/451ebffc-7a95-491f-9c6b-6f508bc584d1 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:44:12.799920 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-45d68176-722c-4fb1-9a56-8616e42aa465 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:12.800524 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-45d68176-722c-4fb1-9a56-8616e42aa465 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:12.858521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-45d68176-722c-4fb1-9a56-8616e42aa465 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:12.866181 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-45d68176-722c-4fb1-9a56-8616e42aa465 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1215 microversion: 2.1 time: 0.072540 Oct 05 14:44:12.866757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 272/528] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:12 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1215 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:13.889127 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:13.889871 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:13.900304 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:13.901026 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:13.901723 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:13.902675 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:13.903574 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:13.904220 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:13.976744 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:13.986493 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8973106d-3bc4-4a67-af8b-739ee8b170fa tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1219 microversion: 2.1 time: 0.100343 Oct 05 14:44:13.987779 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 273/529] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:13 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:15.013526 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:15.015149 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:15.026965 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:15.027947 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:15.028583 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:15.029540 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:15.039591 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:15.040786 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:15.157737 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:15.165953 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-0c1f8d72-395c-43cd-b2d8-a3b2b07a689e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1219 microversion: 2.1 time: 0.158887 Oct 05 14:44:15.166530 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 274/530] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:15 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1219 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:16.190295 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:16.190838 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:16.208131 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:16.208627 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:16.209565 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:16.211327 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:16.217286 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:16.217803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:16.326253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:16.347470 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-535490a0-af80-4cca-8dc7-101006142475 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1285 microversion: 2.1 time: 0.161836 Oct 05 14:44:16.348452 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 275/531] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:16 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1285 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:17.374679 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:17.375416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:17.385023 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:17.385544 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:17.389441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:17.390381 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:17.391492 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:17.392330 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:17.399005 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:17.563496 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:17.574316 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8687b08a-e6b6-4783-bbb7-a9ea074aaeac tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.203437 Oct 05 14:44:17.575683 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 276/532] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:17 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:18.084969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-1b4a7764-562d-451b-8d04-e8c060847a20 req-d45bf514-af77-4f49-97f6-107c532ddef6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "336ef880-f175-4500-824c-5230133c1e69", "tag": "8c806104-7fca-492f-beeb-3ccc9eb9a2b9"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:44:18.098327 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-1b4a7764-562d-451b-8d04-e8c060847a20 req-d45bf514-af77-4f49-97f6-107c532ddef6 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:18.099225 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-1b4a7764-562d-451b-8d04-e8c060847a20 req-d45bf514-af77-4f49-97f6-107c532ddef6 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:18.100290 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-1b4a7764-562d-451b-8d04-e8c060847a20 req-d45bf514-af77-4f49-97f6-107c532ddef6 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:18.115109 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-1b4a7764-562d-451b-8d04-e8c060847a20 req-d45bf514-af77-4f49-97f6-107c532ddef6 service nova] Creating event network-changed:8c806104-7fca-492f-beeb-3ccc9eb9a2b9 for instance 336ef880-f175-4500-824c-5230133c1e69 on hv-cinder-83587 Oct 05 14:44:18.121915 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-1b4a7764-562d-451b-8d04-e8c060847a20 req-d45bf514-af77-4f49-97f6-107c532ddef6 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041677 Oct 05 14:44:18.123012 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 277/533] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:44:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:44:18.599862 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:18.600663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:18.607264 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:18.607752 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:18.608205 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:18.608895 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:18.609401 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:18.609792 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:18.777014 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:18.789562 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2b9d4389-ece8-4ff5-bf21-5dd09a98893a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.194594 Oct 05 14:44:18.790173 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 278/534] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:18 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:19.812253 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:19.813708 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:19.821426 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:19.821940 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:19.822371 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:19.823063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:19.823547 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:19.824606 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:19.993769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:20.001747 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-946f9137-e718-490c-aea2-f448fd45ae2a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.193501 Oct 05 14:44:20.002416 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 279/535] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:19 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:21.025308 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:21.026120 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:21.033008 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:21.033478 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:21.033941 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:21.034622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:21.035064 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:21.035472 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:21.175105 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:21.183493 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f1b269f1-1149-49b2-b36a-e2b381ba14ca tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.163102 Oct 05 14:44:21.184116 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 280/536] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:21 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:21.523699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:44:21.524196 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:44:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:44:22.206101 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:22.206698 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:22.209107 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:44:22.209250 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:44:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:44:22.216506 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:22.216988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:22.217401 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:22.218116 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:22.218565 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:22.218988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:22.351691 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:22.364751 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-78f3a09e-6058-4556-bc29-83f239ed8648 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.162268 Oct 05 14:44:22.365609 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 281/537] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:22 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:23.397298 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:23.398039 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:23.399909 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=2 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:23.400723 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell0:SELECT=8 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:23.406975 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:23.407456 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:23.408066 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:23.408765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:23.409214 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:23.409628 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:23.543924 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:23.552314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f0e4d471-7efb-4e10-9fbd-878a0b599ef3 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.159531 Oct 05 14:44:23.557748 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 282/538] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:23 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:24.576722 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:24.578634 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:24.585775 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:24.586645 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:24.587298 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:24.588167 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:24.588887 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:24.589430 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:24.745234 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:24.755767 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-aae16709-563f-4bb6-b889-597c075a054f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.182897 Oct 05 14:44:24.756405 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 283/539] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:24 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:25.778486 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:25.779334 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:25.785577 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:25.786077 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:25.786498 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:25.787200 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:25.787681 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:25.788061 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:25.990317 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:25.998234 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6c47eb99-6da8-4310-a372-075622a16777 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.223561 Oct 05 14:44:25.998767 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 284/540] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:25 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:27.020617 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:27.021226 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:27.027591 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:27.028065 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:27.028655 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:27.029393 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:27.029860 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:27.030275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:27.189324 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:27.198722 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b9102efd-0f53-4a54-9c52-21f00a6c8d74 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.181741 Oct 05 14:44:27.207714 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 285/541] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:27 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:28.221428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:28.222062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:28.228863 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:28.229336 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:28.229836 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:28.230520 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:28.231004 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:28.231422 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:28.360652 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:28.368490 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-4cc876b2-9f59-4492-8a62-bc13fb250199 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.150970 Oct 05 14:44:28.369059 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 286/542] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:28 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:29.395002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:29.395971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:29.405935 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:29.406400 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:29.406826 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:29.407539 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:29.407956 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:29.408374 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:29.539407 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:29.548020 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-2ce6c18d-f58d-4621-8e27-a1d5dec8d54f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.159661 Oct 05 14:44:29.548587 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 287/543] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:29 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:30.571578 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:30.572202 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:30.578865 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:30.579347 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:30.579767 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:30.580538 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:30.580981 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:30.581465 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:30.717265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:30.724903 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-795f7434-add7-4e1b-9c9e-24a10f007e4f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.157451 Oct 05 14:44:30.725518 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 288/544] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:30 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:31.398653 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8e05cc9b-fdbc-4431-8de9-ef263828e5d4", "device": "/dev/sdb"}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:44:31.401576 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:31.403569 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:31.409269 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:31.409699 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:31.410137 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:31.410820 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:31.411257 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:31.411664 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:35.147799 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-96945445-0882-4873-b15e-ee268251217b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "POST /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.752872 Oct 05 14:44:35.153663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 289/545] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 5 14:44:31 2022] POST /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments => generated 194 bytes in 3755 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:44:35.154874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:35.155615 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:35.164786 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:35.165275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:35.165730 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:35.166726 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:35.167257 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:35.168082 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:35.302855 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:35.310909 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-20d06756-7e9c-43d8-8900-5388236453b6 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.161112 Oct 05 14:44:35.311452 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 290/546] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:35 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:35.602189 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-3e7bb887-e643-49fc-b015-280bbef9c806 req-b7e831f2-ab6a-47ab-beeb-62655f9c87f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "336ef880-f175-4500-824c-5230133c1e69", "name": "network-vif-plugged", "status": "completed", "tag": "8c806104-7fca-492f-beeb-3ccc9eb9a2b9"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:44:35.613484 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-3e7bb887-e643-49fc-b015-280bbef9c806 req-b7e831f2-ab6a-47ab-beeb-62655f9c87f8 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:35.614027 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-3e7bb887-e643-49fc-b015-280bbef9c806 req-b7e831f2-ab6a-47ab-beeb-62655f9c87f8 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:35.614449 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-3e7bb887-e643-49fc-b015-280bbef9c806 req-b7e831f2-ab6a-47ab-beeb-62655f9c87f8 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:35.626049 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-3e7bb887-e643-49fc-b015-280bbef9c806 req-b7e831f2-ab6a-47ab-beeb-62655f9c87f8 service nova] Creating event network-vif-plugged:8c806104-7fca-492f-beeb-3ccc9eb9a2b9 for instance 336ef880-f175-4500-824c-5230133c1e69 on hv-cinder-83587 Oct 05 14:44:35.631777 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-3e7bb887-e643-49fc-b015-280bbef9c806 req-b7e831f2-ab6a-47ab-beeb-62655f9c87f8 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033404 Oct 05 14:44:35.632543 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 291/547] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:44:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:44:36.335785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:36.336474 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:36.343543 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:36.344017 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:36.344438 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:36.345115 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:36.345554 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:36.345988 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:36.481028 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:36.484625 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:36.490798 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-56949ebb-258b-462c-af89-9fb07b0aa479 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.160002 Oct 05 14:44:36.491360 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 292/548] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:36 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:37.517757 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:37.518368 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:37.526859 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:37.527348 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:37.527774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:37.528473 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:37.529196 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:37.529721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:37.690776 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:37.698748 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-38a57f81-9a2b-4348-826d-ab775b79c70e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1315 microversion: 2.1 time: 0.185887 Oct 05 14:44:37.699344 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 293/549] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:37 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1315 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:38.721178 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:38.721809 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:38.729077 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:38.729559 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:38.730015 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:38.730782 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:38.731102 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:38.737292 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:38.892805 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:38.901233 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-6e1dc4df-e462-4742-a942-1d66490ae40a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1464 microversion: 2.1 time: 0.183760 Oct 05 14:44:38.901947 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 294/550] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:38 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1464 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:38.957311 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:38.958090 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:38.964781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:38.965336 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:38.965874 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:38.966638 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:38.967155 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:38.967572 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:38.993954 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:44:39.078028 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-67f44008-3661-4b57-9c4d-d8b489a3a818 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "DELETE /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 204 len: 0 microversion: 2.1 time: 0.127691 Oct 05 14:44:39.079142 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 295/551] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:44:38 2022] DELETE /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 05 14:44:39.089803 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:39.093907 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:39.099994 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:39.100482 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:39.100930 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:39.101609 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:39.102075 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:39.102481 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:39.247164 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:39.255501 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c533704b-b6b0-47c7-92b2-13ae85938506 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1470 microversion: 2.1 time: 0.171658 Oct 05 14:44:39.256054 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 296/552] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:39 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1470 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:44:40.283199 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:44:40.283809 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:44:40.293441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:40.293441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:40.293441 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:40.294626 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:44:40.295519 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:44:40.296321 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:44:40.433678 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '005d0f9318534cfc98c37d822e49bdc4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:44:40.443193 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e67c7f13-af71-4d35-9559-6a14daa2c3f0 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 200 len: 1470 microversion: 2.1 time: 0.164147 Oct 05 14:44:40.443790 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 297/553] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:44:40 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 1470 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:44:40.788457 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-aeb87f64-fce1-4e02-b70e-5031fed6e6a1 tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-188791589", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-648941858", "security_groups": [{"name": "tempest-secgroup-smoke-503463352"}], "block_device_mapping_v2": [{"uuid": "234cec89-f68f-4d35-9750-5244fbfef244", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "8bc08524-37ee-46a4-9444-4e7e9b24d3aa"}]}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:44:50.438047 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:50.809498 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=24,nova_api:UPDATE=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:47:41.535123 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:47:41.535708 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:47:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:47:42.219068 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:47:42.220034 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:47:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:47:42.227236 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:47:42.227472 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:47:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:49:21.541152 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:49:21.541690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:49:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:49:21.542552 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:49:21.542721 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:49:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:49:22.224622 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:49:22.225561 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:49:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:51:01.550948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:51:01.551465 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:51:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:51:02.235922 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:51:02.236905 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:51:02 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:51:02.238781 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:51:02.239035 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:51:02 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:52:41.559402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:52:41.560012 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:52:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:52:41.561257 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:52:41.561403 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:52:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:52:42.239898 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:52:42.240517 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:52:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:54:21.570682 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:54:21.571523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:54:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:54:22.250640 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:54:22.251558 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:54:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:54:22.253506 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:54:22.254021 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:54:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:56:01.580744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:56:01.581658 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:56:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:56:01.583080 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:56:01.583228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:56:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:56:02.253831 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:56:02.254358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:56:02 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:57:41.591518 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:57:41.591996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:57:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:57:42.267929 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:57:42.268927 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:57:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:57:42.271440 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:57:42.272061 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:57:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:58:56.811504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG neutronclient.v2_0.client [None req-80e1685a-35b4-42da-a318-3d0c35163667 tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Error message: {"error": {"code": 401, "title": "Unauthorized", "message": "The request you have made requires authentication."}} {{(pid=120605) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 05 14:58:56.812300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-80e1685a-35b4-42da-a318-3d0c35163667 tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 500 len: 0 microversion: 2.1 time: 898.844842 Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack [None req-80e1685a-35b4-42da-a318-3d0c35163667 tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Caught error: Not authorized.: nova.exception.Unauthorized: Not authorized. Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Traceback (most recent call last): Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 476, in find_resource Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.find_resource_by_id(resource, name_or_id, Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 440, in find_resource_by_id Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack raise exceptions.NotFound(message=not_found_message) Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack neutronclient.common.exceptions.NotFound: Unable to find security_group with id 'tempest-secgroup-smoke-1011708186' Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack During handling of the above exception, another exception occurred: Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Traceback (most recent call last): Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack exception_handler_v20(status_code, error_body) Oct 05 14:58:56.852779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack raise client_exc(message=error_message, Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack neutronclient.common.exceptions.Unauthorized: 401-{'error': {'code': 401, 'title': 'Unauthorized', 'message': 'The request you have made requires authentication.'}} Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Neutron server returns request_ids: ['req-b8d384d6-4259-49ba-bac9-d538c108945d'] Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack During handling of the above exception, another exception occurred: Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Traceback (most recent call last): Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/__init__.py", line 95, in __call__ Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return req.get_response(self.application) Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 104, in __call__ Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack self._log_req(req, res, start) Oct 05 14:58:56.854356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack self.force_reraise() Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack raise self.value Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 99, in __call__ Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack res = req.get_response(self.application) Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/osprofiler/web.py", line 111, in __call__ Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return request.get_response(self.application) Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:58:56.856010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack response = req.get_response(self._app) Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/routes/middleware.py", line 153, in __call__ Oct 05 14:58:56.857656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack response = self.app(environ, start_response) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 486, in __call__ Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self._process_stack(request, action, action_args, Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return method(req=request, **action_args) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return f(*args, **kwargs) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 05 14:58:56.859113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack [Previous line repeated 10 more times] Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack instances, resv_id = self.compute_api.create( Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self._create_instance( Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ) = self._validate_and_build_base_options( Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1074, in _validate_and_build_base_options Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack security_groups = self._check_requested_secgroups( Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 521, in _check_requested_secgroups Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack secgroup_uuid = security_group_api.validate_name(context, secgroup) Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/security_group_api.py", line 68, in validate_name Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return neutronv20.find_resourceid_by_name_or_id( Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 56, in find_resourceid_by_name_or_id Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return find_resource_by_name_or_id(client, resource, name_or_id, Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 49, in find_resource_by_name_or_id Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return client.find_resource(resource, name_or_id, project_id, Oct 05 14:58:56.860744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 480, in find_resource Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self._find_resource_by_name( Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 457, in _find_resource_by_name Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack data = obj_lister(**params) Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1063, in list_security_groups Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.list('security_groups', self.security_groups_path, Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 372, in list Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack for r in self._pagination(collection, path, **params): Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 387, in _pagination Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack res = self.get(path, params=params) Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.862396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.retry_request("GET", action, body=body, Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack return self.do_request(method, action, body=body, Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack self._handle_fault_response(status_code, replybody, resp) Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 204, in wrapper Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack raise exception.Unauthorized() Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack nova.exception.Unauthorized: Not authorized. Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack Oct 05 14:58:56.863999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack [None req-80e1685a-35b4-42da-a318-3d0c35163667 tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] http://10.222.0.23/compute/v2.1/servers returned with HTTP 401 Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-80e1685a-35b4-42da-a318-3d0c35163667 tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Returning 401 to user: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/servers (10.222.0.23) Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-80e1685a-35b4-42da-a318-3d0c35163667 tempest-TestShelveInstance-1249654648 tempest-TestShelveInstance-1249654648-project] Unhandled error: OSError: write error Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 257/554] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:43:57 2022] POST /compute/v2.1/servers => generated 0 bytes in 898889 msecs (HTTP/1.1 401) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:56.865243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:56.866003 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:56.867684 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:56.868163 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:56.868586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:56.869286 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:56.869755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:56.870168 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:56.883799 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:56.885295 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:56.887225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:56.888284 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:56.890044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.030136 Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-ae832a30-d20f-4b43-9077-bb95235f0e2f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:56.891573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 258/555] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:56 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:56.950307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "336ef880-f175-4500-824c-5230133c1e69", "name": "network-vif-deleted", "tag": "8c806104-7fca-492f-beeb-3ccc9eb9a2b9"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:58:56.960199 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:56.960636 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:56.961047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:56.972221 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Creating event network-vif-deleted:8c806104-7fca-492f-beeb-3ccc9eb9a2b9 for instance 336ef880-f175-4500-824c-5230133c1e69 on hv-cinder-83587 Oct 05 14:58:56.978910 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090134 Oct 05 14:58:56.979422 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.23) !!! Oct 05 14:58:56.979422 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.23) Oct 05 14:58:56.980184 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Unhandled error: OSError: write error Oct 05 14:58:56.980184 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:56.980184 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:56.980496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 259/556] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:58:56 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:56.985949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:56.987550 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.005326 Oct 05 14:58:56.987918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:56.987918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:56 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:56.988519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [req-05aa525c-cfd8-418f-b4a7-e99125dce7e1 req-cf47b478-e0c4-401d-bb02-240d6384a77c service nova] Unhandled error: OSError: write error Oct 05 14:58:56.988519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:56.988519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:56.988519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 260/557] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 8 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:56.993510 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:56.994103 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:57.000548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.001014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.001448 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.002178 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.002616 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.003025 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.142821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lazy-loading 'services' on Instance uuid 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:57.258489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 202 len: 0 microversion: 2.1 time: 0.268822 Oct 05 14:58:57.259008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:57.259008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:57.259688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:57.259688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.259688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.259982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 261/558] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:56 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 271 msecs (HTTP/1.1 202) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.307239 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken: Token authorization failed Oct 05 14:58:57.307998 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 401 len: 114 microversion: - time: 0.047095 Oct 05 14:58:57.308623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:58:57.308623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/servers (10.222.0.23) Oct 05 14:58:57.309180 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-ff45c344-4e2e-4911-a87b-a3702a8488a4 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:57.309180 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.309180 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.309470 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 262/559] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:58:57 2022] POST /compute/v2.1/servers => generated 0 bytes in 49 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.314012 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:57.314686 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:57.321889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.322370 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.322798 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.323542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.323990 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.324503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.338454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:57.340137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.341870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:57.343024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.344330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.031323 Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.345485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 263/560] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:57 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.391496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken: Token authorization failed Oct 05 14:58:57.391496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.047270 Oct 05 14:58:57.391496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:57.391496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:57.392157 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-b77da902-8167-4bf1-9cce-ea632d9b3f47 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:57.392157 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.392157 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.392157 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 264/561] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 49 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.396449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:57.396933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:57.402956 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.403433 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.403883 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.404545 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.404997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.405412 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.485950 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'services' on Instance uuid 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:57.570344 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 202 len: 0 microversion: 2.1 time: 0.177212 Oct 05 14:58:57.570875 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:57.570875 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:57.571566 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:57.571566 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.571566 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.571886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 265/562] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 179 msecs (HTTP/1.1 202) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.574202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:57.574999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.002143 Oct 05 14:58:57.575435 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:57.575435 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:57.575995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-48c8f3af-676a-40e0-9ee4-4a64f77e2018 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:57.575995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.575995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.576261 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 266/563] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 4 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.580812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:57.581384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:57.588237 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.588740 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.589171 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.589895 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.590432 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.590884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.673668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lazy-loading 'services' on Instance uuid 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:57.732549 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:57.734822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:57.736451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:57.737189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:57.738444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.739796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.741011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) Oct 05 14:58:57.742304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.743455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f8707d7-5de0-4cb2-876e-1a657ed5554c) {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:57.743455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 400 len: 269 microversion: 2.1 time: 0.159312 Oct 05 14:58:57.743455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:57.743455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:57.743455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:57.743455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.743455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 267/564] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 161 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 401 len: 114 microversion: - time: 0.001911 Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/servers (10.222.0.23) Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-2f9875b2-4006-4278-9209-4740ab2e6f41 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.744061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 268/565] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:58:57 2022] POST /compute/v2.1/servers => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.745246 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:57.745901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:57.754966 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.755447 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.755872 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.756546 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.756989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.757402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.770653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:57.772060 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.773709 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:57.774615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:57.775957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.031469 Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 269/566] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:57 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.777121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:57.777829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001830 Oct 05 14:58:57.777829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:57.777829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:57.778075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-2f61f075-251a-42c4-9ed1-5a0031314bd2 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:57.778075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.778075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.778075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 270/567] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.782363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:57.782828 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:57.788535 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.789007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.789430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.790121 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.790558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.790960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.872297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'services' on Instance uuid 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:57.929772 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:57.931938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:57.933615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:57.934758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:57.936154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.938297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.942009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) Oct 05 14:58:57.945077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:57.946515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a6b628f-91e7-4b76-a8a0-a2aa99e0734c) {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:57.946515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 400 len: 269 microversion: 2.1 time: 0.154409 Oct 05 14:58:57.946515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:57.946515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:57.946515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:57.946515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.946515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 271/568] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 156 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.002609 Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:57 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-99bc6a89-dca2-48f6-9830-9da1e5c3642a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:57.947151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 272/569] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 4 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:57.947923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:57.947923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:57.951463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.951918 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.952351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:57.953039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:57.953483 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:57.953936 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.035157 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lazy-loading 'services' on Instance uuid 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:58.091322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.093499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:58.095101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.095888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.097192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.098699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.099988 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) Oct 05 14:58:58.101364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.102728 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2c47304e-44fa-4d97-a7d0-28f186d206e1) {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.102728 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 400 len: 269 microversion: 2.1 time: 0.155000 Oct 05 14:58:58.102728 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:58.102728 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:58.102728 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-3238f58b-ce7e-4f35-9f84-69b6a8696920 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:58.102728 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.102728 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.103313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 273/570] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:57 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 156 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.103313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.103313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.106957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.107429 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.107856 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.108536 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.108980 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.109484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.122950 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.124413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.126143 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.127153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.128726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.028771 Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-f6f2e334-5087-4fc1-a5bf-00d3095a973a tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.130208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 274/571] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:58 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.265490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.266017 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.272279 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.272754 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.273177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.273887 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.274331 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.274761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.288317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.289838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.291569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.292656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.294197 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.163505 Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (10.222.0.23) Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.295804 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 275/572] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 165 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001937 Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-0b325c24-54d2-418a-b842-f08f302a1006 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.296501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 276/573] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.300444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.300915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.306801 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.307281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.307707 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.308388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.308831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.309321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.391859 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'services' on Instance uuid 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:58.449018 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.451373 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:58.453038 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.453861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.455195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.456666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.458010 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) Oct 05 14:58:58.459392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.461074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d95d643f-cabf-42fb-952b-a832db863e3d) {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.461074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 400 len: 269 microversion: 2.1 time: 0.155830 Oct 05 14:58:58.461074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:58.461074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:58.461074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:58.461074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.461074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 277/574] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 157 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.001954 Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-ec45e925-dbb3-4698-a1a7-220f45baf9d8 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.461705 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 278/575] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.462478 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.462881 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.469462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.469983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.470411 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.471083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.471556 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.472066 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.549739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lazy-loading 'services' on Instance uuid 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:58.603553 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.605781 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:58.607531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.608628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.610098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.611594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.612849 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) Oct 05 14:58:58.614327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.616529 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-33a4ac27-78d7-4923-9312-a67925defa96) {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.617817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 400 len: 269 microversion: 2.1 time: 0.158883 Oct 05 14:58:58.618378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:58.618515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:58.619016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-a8f050d9-ba10-4261-b6bc-72038745c47f tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:58.619016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.619016 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.619407 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 279/576] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 161 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.625666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.626289 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.632445 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.632947 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.633370 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.634147 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.634599 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.635018 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.649251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.650926 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.652388 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.653281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.655238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.031948 Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-0f6d7ddd-cd4c-4002-805d-9d94144bac3e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.657390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 280/577] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:58 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.658462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.658611 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.665062 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.665510 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.665979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.666651 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.667092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.667539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.680771 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.682300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.683997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.687198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:58.689292 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.029135 Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (10.222.0.23) Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.692172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 281/578] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.002084 Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-4b1804dd-7a3c-47b5-8a80-d5d032bdeccc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.692861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 282/579] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 4 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.694935 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.695411 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.701227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.701881 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.702297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.702965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.703396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.703800 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.785491 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'services' on Instance uuid 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:58.840999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.843207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:58.844690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.845360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:58.846513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.847850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.849013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) Oct 05 14:58:58.850243 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.851505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-81535315-f05b-4abb-8c69-29395f7620c6) {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.851505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 400 len: 269 microversion: 2.1 time: 0.153830 Oct 05 14:58:58.851505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:58.851505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:58.851505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:58.851505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.851505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 283/580] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 156 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.001976 Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-b0dc33fe-14d1-45a5-9676-120200105e6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.852024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 284/581] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.853938 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.854397 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.860958 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.861437 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.861901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.862576 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.863019 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.863429 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.940578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.942900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.944484 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8e05cc9b-fdbc-4431-8de9-ef263828e5d4. Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.945352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.946983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.948281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 404 len: 154 microversion: 2.1 time: 0.093930 Oct 05 14:58:58.948281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:58.948281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:58.948281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-e8be2474-4f67-472a-a3c0-304b9a333729 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:58.948281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.948281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.948281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 285/582] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.949800 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.950400 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.955863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.956334 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.956759 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.957430 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.957931 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.958341 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.972263 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:58.973780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.975401 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:58.976406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:58.978071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.028797 Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:58 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-8684ffb0-87a2-4cf3-9569-13b1c3be3754 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:58.979463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 286/583] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:58 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:58.980220 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:58.980689 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:58.986012 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.986483 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.986889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:58.987558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:58.987997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:58.988408 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.002291 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.003761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.005446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.006427 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.007952 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.028114 Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (10.222.0.23) Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.009584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 287/584] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:58:58 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001846 Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-67f75687-dbc3-4f8d-8c71-535eec244c4d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.010297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 288/585] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.014130 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.014602 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.020369 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.020831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.021280 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.021995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.022444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.022891 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.102105 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:58:59.106698 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'services' on Instance uuid 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:58:59.162132 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.164307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.165775 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:59.166558 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:58:59.167681 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.168994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.170139 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) Oct 05 14:58:59.171304 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.172436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5753ab45-7bb7-4ffd-9e89-945d0aa0a05d) {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.172436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 400 len: 269 microversion: 2.1 time: 0.155067 Oct 05 14:58:59.172436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:59.172436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:59.172436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.172436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.172436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 289/586] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 156 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.002070 Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-a5cf2fa4-0722-4e0d-8889-892d65b3e473 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.172948 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 290/587] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.175200 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.175682 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.182605 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.183169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.183756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.184583 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.185075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.185514 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.264875 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.266384 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.267995 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8e05cc9b-fdbc-4431-8de9-ef263828e5d4. Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.268739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.270063 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.271091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 404 len: 154 microversion: 2.1 time: 0.096158 Oct 05 14:58:59.271091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:59.271091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:59.271091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-9b13b8d4-eafa-4dbe-bba9-fe542264831c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:59.271091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.271091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.271091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 291/588] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.273257 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.273889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.279610 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.280093 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.280525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.281202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.281676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.282092 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.295586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.297064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.298927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.299909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.301473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.028673 Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-a3c16e93-0f76-40bf-82af-976f132c6b7e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.302989 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 292/589] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:59 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.303679 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.304083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.309488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.309931 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.310355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.311049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.311494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.311911 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.325410 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.326925 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.328641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.329624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.331409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.027941 Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (10.222.0.23) Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.333029 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 293/590] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001860 Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-3b32bd47-e063-419f-8062-1252170b0e1f tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.333784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 294/591] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.337674 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.338172 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.343683 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.344302 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.344580 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.345259 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.345706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.346149 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.426177 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.427671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.429319 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.430258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.431666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.095266 Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 295/592] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.432720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.433307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.001883 Oct 05 14:58:59.433426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:59.433426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:59.433975 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-dfef4acc-84c6-4196-8241-fa4ed0e6cff3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.433975 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.433975 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.434262 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 296/593] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.438535 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.439012 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.444840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.445320 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.445778 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.446451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.446890 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.447303 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.529764 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.531253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.533001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8e05cc9b-fdbc-4431-8de9-ef263828e5d4. Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.533927 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.535393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.536578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 404 len: 154 microversion: 2.1 time: 0.097697 Oct 05 14:58:59.536578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:59.536578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:59.536578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-fa2ea10a-75c7-45fc-a738-d1c237b7918a tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:59.536578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.536578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.536578 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 297/594] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.538164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.538766 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.544368 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.544857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.545282 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.546006 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.546453 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.546864 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.562403 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.563811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.565449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.566504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.568039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.030513 Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-93e7ca2f-80c0-4844-a2f5-ac23ffcad371 tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.569504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 298/595] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:59 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.570322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.570820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.576351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.576820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.577248 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.577945 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.578392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.578812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.592274 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.593949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.595824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.596783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.598496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.028287 Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (10.222.0.23) Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.600054 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 299/596] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001908 Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-e280f85c-1f6c-4051-9a58-caed75094745 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.600788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 300/597] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.604486 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.604964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.610534 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.611032 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.611462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.612131 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.612580 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.612997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.686889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.688377 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.690050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.690837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.692524 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.088744 Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 301/598] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.693768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.694442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.001938 Oct 05 14:58:59.694442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:59.694442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:59.694704 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-ff8255fa-0ffa-42eb-b8d0-813cf2c57dc7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.694704 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.694704 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.694704 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 302/599] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.699125 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.699645 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.705267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.705802 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.706230 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.706907 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.707352 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.707767 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.790863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.793153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.794900 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8e05cc9b-fdbc-4431-8de9-ef263828e5d4. Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.795812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.797379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.798657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 404 len: 154 microversion: 2.1 time: 0.099059 Oct 05 14:58:59.798657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:58:59.798657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:58:59.798657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-24e68ad8-f3c6-4d7c-a6a4-e52b4e62f9d5 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:58:59.798657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.798657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.798657 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 303/600] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.800729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.801673 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.810353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.810850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.811279 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.811999 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.812514 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.812866 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.826421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.827901 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.829629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.830719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:58:59.832365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.032598 Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-b2ffd0a1-5e53-4d62-b3e4-c2056808e24f tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.833729 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 304/601] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:58:59 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.835817 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.836293 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.841941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.842406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.842824 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.843482 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.843917 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.844328 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.857381 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.858937 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.860653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.861668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:58:59.863223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.028654 Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (10.222.0.23) Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.864812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 305/602] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001874 Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-f80e78e1-a6ed-45c6-b425-54bc17b3591d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.865490 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 306/603] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.869239 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.869732 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.875390 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.875865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.876285 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.876949 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.877391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.877841 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.954126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:58:59.955606 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:58:59.957342 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:58:59.958301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:58:59.960182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.091167 Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 307/604] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.961359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:58:59.962053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.002221 Oct 05 14:58:59.962053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:58:59.962053 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:58:59 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:58:59.962328 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-950df0b3-d75c-462a-8e1c-92397c46698b tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:58:59.962328 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:58:59.962328 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:58:59.962328 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 308/605] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 4 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:58:59.966676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:58:59.967156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:58:59.972809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.973252 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.973687 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:58:59.974367 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:58:59.974777 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:58:59.975183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.052278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.053861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.055615 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8e05cc9b-fdbc-4431-8de9-ef263828e5d4. Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.056523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.058113 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.059323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 404 len: 154 microversion: 2.1 time: 0.092184 Oct 05 14:59:00.059323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (ip 10.222.0.23) !!! Oct 05 14:59:00.059323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 (10.222.0.23) Oct 05 14:59:00.059323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-59507083-90b1-4fb0-a15d-41f607efe694 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Unhandled error: OSError: write error Oct 05 14:59:00.059323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:59:00.059323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:59:00.059323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 309/606] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:58:59 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 0 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:59:00.060855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.061469 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.067169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.067616 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.068046 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.068726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.069169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.069567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.084376 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.086748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.088413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.089386 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.091212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.030885 Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (ip 10.222.0.23) !!! Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 (10.222.0.23) Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-7d29f6d7-47b7-4be9-bcbf-86b1424a84ec tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Unhandled error: OSError: write error Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:59:00.092665 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 310/607] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:00 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 0 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:59:00.093335 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.093891 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.099696 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.100173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.100599 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.101267 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.101740 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.102158 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.115320 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.116795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.118560 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.119527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.121055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.028138 Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (ip 10.222.0.23) !!! Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 (10.222.0.23) Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:59:00.122784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 311/608] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 0 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001846 Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (ip 10.222.0.23) !!! Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 (10.222.0.23) Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-f6230555-069c-44d8-ae47-a294a10bacf0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Unhandled error: OSError: write error Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:59:00.123467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 312/609] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:59:00.127117 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.127588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.132857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.133329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.133796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.134460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.134941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.135356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.210064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.211561 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.213247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.214202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.215442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.089268 Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (ip 10.222.0.23) !!! Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d (10.222.0.23) Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 313/610] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 0 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 05 14:59:00.216515 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:59:00.217077 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.002082 Oct 05 14:59:00.217421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (ip 10.222.0.23) !!! Oct 05 14:59:00.217421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 14:59:00 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 (10.222.0.23) Oct 05 14:59:00.218009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: CRITICAL nova [None req-a4b987c7-34a5-4e91-ac33-b9cc907ec736 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Unhandled error: OSError: write error Oct 05 14:59:00.218009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova OSError: write error Oct 05 14:59:00.218009 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova Oct 05 14:59:00.218317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 314/611] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Oct 05 14:59:00.222514 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.222993 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.228663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.229148 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.229627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.230337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.230777 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.231185 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.322089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.324236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.325867 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8e05cc9b-fdbc-4431-8de9-ef263828e5d4. Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.326623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.327915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.328959 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-49d9c794-c082-4270-8c07-481f61d8dd02 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 404 len: 154 microversion: 2.1 time: 0.106750 Oct 05 14:59:00.328959 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 315/612] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 154 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.331556 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.332329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] [instance: 336ef880-f175-4500-824c-5230133c1e69] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.341280 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.341776 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.342200 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.342886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.343324 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.343746 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.357147 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] HTTP exception thrown: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Request method failure captured: Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 HTTP/1.1 Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgyeKu2phntWUlrzTnrxxc8cLkfYfXfhprXNcjrHXsXeRWUuRQ0zy5LrZypFFIi2CT7nOOPtcI7kKBRpuImF2NkN9ENLmjWLY7umG5hd_Jy12JH242sRwPcbCtfzMVkEvDQRve5LcSvuvdGOsQkU2h_LBpgw44t3YBvHPSUS8uWSto Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.358676 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_bea771b3ff124fc8b9b9ac57f8ea12b4", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/bea771b3ff124fc8b9b9ac57f8ea12b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesNegativeTest-672425805 Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: bea771b3ff124fc8b9b9ac57f8ea12b4 Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesNegativeTest-672425805 Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesNegativeTest-672425805-project Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 005d0f9318534cfc98c37d822e49bdc4 Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesNegativeTest-672425805-project Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.360428 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '336ef880-f175-4500-824c-5230133c1e69'} Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.361402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. Oct 05 14:59:00.363141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.364779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] Returning 404 to user: Instance 336ef880-f175-4500-824c-5230133c1e69 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.364779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b87795e0-7033-41d4-a6c2-573f79dde06e tempest-VolumesNegativeTest-672425805 tempest-VolumesNegativeTest-672425805-project] 10.222.0.23 "GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69" status: 404 len: 111 microversion: 2.1 time: 0.032590 Oct 05 14:59:00.364779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 316/613] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:00 2022] GET /compute/v2.1/servers/336ef880-f175-4500-824c-5230133c1e69 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.364779 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.365169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.370544 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.371011 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.371442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.372165 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.372608 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.373021 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.385928 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.387392 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.389299 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.390475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.392523 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.394232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.394232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.027388 Oct 05 14:59:00.394232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 317/614] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 111 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.394232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:59:00.394232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1ad46556-3bbe-484b-97ec-7430f108f96e tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001886 Oct 05 14:59:00.394232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 318/615] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 114 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 293 bytes (1 switches on core 0) Oct 05 14:59:00.396905 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.397371 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.402689 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.403168 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.403667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.404322 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.404764 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.405173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.479667 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.481136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.482794 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.483585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.484880 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.485981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.089072 Oct 05 14:59:00.485981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 319/616] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 154 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.485981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:59:00.485981 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-057a303f-7696-45fd-84f2-b28fcb4c30ee tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331" status: 401 len: 114 microversion: - time: 0.001883 Oct 05 14:59:00.486452 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 320/617] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1870409331 => generated 114 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 293 bytes (1 switches on core 0) Oct 05 14:59:00.490905 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.491385 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.497181 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.497678 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.498105 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.498787 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.499225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.499635 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.574656 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 HTTP/1.1 Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.576078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.577840 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633', 'id': '8e05cc9b-fdbc-4431-8de9-ef263828e5d4'} Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 8e05cc9b-fdbc-4431-8de9-ef263828e5d4. Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.578892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.580378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 is not attached to volume 8e05cc9b-fdbc-4431-8de9-ef263828e5d4 {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.581582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-7fe0ba42-44d6-4a77-b995-2ad58f45cd74 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4" status: 404 len: 154 microversion: 2.1 time: 0.090107 Oct 05 14:59:00.581582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 321/618] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633/os-volume_attachments/8e05cc9b-fdbc-4431-8de9-ef263828e5d4 => generated 154 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.582446 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.582924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.588612 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.589081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.589501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.590530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.590801 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.591190 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.605874 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.607419 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.609156 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.610175 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.611755 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.613406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.613406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.029777 Oct 05 14:59:00.613406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 322/619] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.613406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: WARNING keystonemiddleware.auth_token [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Oct 05 14:59:00.613406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-23b2ac6b-76aa-4dc1-809d-8dd5f597eddc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858" status: 401 len: 114 microversion: - time: 0.001854 Oct 05 14:59:00.613406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 323/620] 10.222.0.23 () {58 vars in 1260 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-648941858 => generated 114 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 293 bytes (1 switches on core 0) Oct 05 14:59:00.616780 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.617251 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.622618 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.623085 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.623509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.624162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.624601 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.625007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.697104 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.698586 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.700628 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.701402 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.702803 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.703831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0e0b579a-dae3-4789-80c0-190aabc3b7e5 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.086555 Oct 05 14:59:00.703831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 324/621] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 154 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.705908 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.706733 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.713217 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.713700 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.714141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.714811 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.715227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.715691 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.729869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.731499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.733224 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.734336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.736081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.737825 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.737825 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-deaa792d-1318-4e0e-88da-733ad0ad89fc tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.031344 Oct 05 14:59:00.737825 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 325/622] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.738108 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.738465 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.743960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.744424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.744845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.745501 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.745970 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.746379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.750020 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:00.822354 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.823834 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.825663 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.826594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.828089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.829314 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3c691dbe-d524-4100-bc27-f76c385ab871 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.090874 Oct 05 14:59:00.829314 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 326/623] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 154 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.830240 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.830720 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.836572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.837049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.837473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.838176 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.838621 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.839032 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.851980 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.853458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.855360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.856520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.858099 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.859660 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.859660 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-875244e7-73b1-4fec-a127-b73cab100111 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.027995 Oct 05 14:59:00.859660 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 327/624] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.859660 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.860082 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.865457 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.865966 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.866385 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.867044 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.867485 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.867892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.942631 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.944466 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.946047 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.946964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.948317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.949323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-7edc0b57-8663-4eb6-8811-932f4e5a6a36 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.089752 Oct 05 14:59:00.949323 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 328/625] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 154 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.951331 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.951924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 41b4c31d-52e5-4475-94d1-d54892780716] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.957526 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.958098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.958444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.959252 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.960052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.960474 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.973479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 HTTP/1.1 Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:00.974982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.976460 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '41b4c31d-52e5-4475-94d1-d54892780716'} Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:00.977310 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. Oct 05 14:59:00.979055 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:00.980363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 41b4c31d-52e5-4475-94d1-d54892780716 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:00.980363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a2485db4-824d-42c5-9595-2565d9d59884 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716" status: 404 len: 111 microversion: 2.1 time: 0.028694 Oct 05 14:59:00.980363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 329/626] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:00 2022] GET /compute/v2.1/servers/41b4c31d-52e5-4475-94d1-d54892780716 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:00.981144 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:00.981662 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:00.987439 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.987912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.988345 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:00.989007 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:00.989444 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:00.989878 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.080891 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d HTTP/1.1 Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:01.082311 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:01.083933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'server_id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7', 'id': '3055bee5-36d3-4d9b-84d8-6d34bb35071d'} Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3055bee5-36d3-4d9b-84d8-6d34bb35071d. Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:01.085096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:01.087442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 is not attached to volume 3055bee5-36d3-4d9b-84d8-6d34bb35071d {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:01.089140 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-cb3fd9a6-6856-4ee6-8956-257991c58ac7 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d" status: 404 len: 154 microversion: 2.1 time: 0.106058 Oct 05 14:59:01.089140 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 330/627] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 14:59:00 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments/3055bee5-36d3-4d9b-84d8-6d34bb35071d => generated 154 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:01.089625 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:01.090090 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:01.093169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:01.097005 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.097506 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.097992 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.098711 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.099262 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.099691 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.127582 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Going to try to terminate instance {{(pid=120605) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:59:01.215192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a98b8b14-f12c-4996-9fa4-f7b4e5847757 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 204 len: 0 microversion: 2.1 time: 0.129153 Oct 05 14:59:01.215876 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 331/628] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:01 2022] DELETE /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:59:01.221286 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:01.223671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:01.229510 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.230026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.230458 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.231148 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.231623 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.232039 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.260790 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-948c5057-c42c-4c69-a610-1e147e44ccbc tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043756 Oct 05 14:59:01.261332 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 332/629] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 5 14:59:01 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 05 14:59:01.265603 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:01.266217 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:01.272313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.272917 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.273238 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.273990 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.274468 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.274879 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.421036 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:01.429153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-84fb1f39-de09-4fd1-8ad3-4d78f1cf6bac tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1513 microversion: 2.1 time: 0.166883 Oct 05 14:59:01.429739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 333/630] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:01 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1513 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:01.434398 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:01.434872 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:01.441671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.442216 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.442666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.443518 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.443983 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.444493 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.470912 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Going to try to terminate instance {{(pid=120605) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 14:59:01.552161 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-52e863e1-004c-4345-9df4-8c14ce4d9173 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 204 len: 0 microversion: 2.1 time: 0.121079 Oct 05 14:59:01.552747 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 334/631] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:01 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 14:59:01.561320 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:01.562037 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:01.567954 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.568432 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.568855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.569540 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:01.570018 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:01.570429 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:01.708822 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:01.717206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0b98c391-f2b7-4f9d-854c-2490599f93b3 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1675 microversion: 2.1 time: 0.159977 Oct 05 14:59:01.717793 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 335/632] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:01 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1675 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:59:02.124153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:02.124741 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:02.130929 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.131406 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.131829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.132503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.132943 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.133355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.148799 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 HTTP/1.1 Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:02.150495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.152193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '26384af3-dd3e-4156-ae91-19e775b78d09'} Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:02.153083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.154509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.155971 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:02.155971 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8c6d2a94-7f44-46e8-9a92-4da61ae1dec0 tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "DELETE /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 404 len: 111 microversion: 2.1 time: 0.031557 Oct 05 14:59:02.155971 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 336/633] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 14:59:02 2022] DELETE /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:02.159652 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:02.160272 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] [instance: 26384af3-dd3e-4156-ae91-19e775b78d09] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:02.166071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.166542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.166960 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.167621 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.168058 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.168470 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.181499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] HTTP exception thrown: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Request method failure captured: Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 HTTP/1.1 Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZCN51saR_QCXiZ8uqofhWQdDi3Ha7uGvXFEK_QBKDAR7B7lljM5qx6-peszSf6l0i0FYwkfkcXMOA9fQZPVIZLehkUTSS--EO-oPbIhdQs9Wb4Dt4lPX1FkX5On4fbCUQe-_noNwIKqtgiLgZRa9JTIpdgQbGa2rs-k7KpnlZgRXH0 Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:02.182984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_45170bafd3c5447cbe6bbb69d7c5f544", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/45170bafd3c5447cbe6bbb69d7c5f544"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 45170bafd3c5447cbe6bbb69d7c5f544 Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-250163126 Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 2e988e0435464c1d8aa4842fab56dfa4 Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeNegativeTest-250163126-project Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.184646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '26384af3-dd3e-4156-ae91-19e775b78d09'} Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:02.185588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. Oct 05 14:59:02.187202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:02.188645 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] Returning 404 to user: Instance 26384af3-dd3e-4156-ae91-19e775b78d09 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:02.188645 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-63895be9-3113-494f-a602-10a600de863d tempest-AttachVolumeNegativeTest-250163126 tempest-AttachVolumeNegativeTest-250163126-project] 10.222.0.23 "GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09" status: 404 len: 111 microversion: 2.1 time: 0.028075 Oct 05 14:59:02.188645 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 337/634] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:02 2022] GET /compute/v2.1/servers/26384af3-dd3e-4156-ae91-19e775b78d09 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:02.452682 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:02.453344 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:02.460242 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.460741 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.461170 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.461892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.462325 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.462753 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.683416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:02.692424 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-08a75d87-e12b-4df7-916a-9834b634493b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1513 microversion: 2.1 time: 0.243736 Oct 05 14:59:02.693796 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 338/635] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:02 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1513 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:02.742877 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:02.743505 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:02.750119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.750379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.750809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.751495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:02.751947 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:02.752358 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:02.984396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:02.993445 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-00b7ad9b-46ed-4af3-b367-864ecad4a452 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1675 microversion: 2.1 time: 0.256115 Oct 05 14:59:02.994059 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 339/636] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:02 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1675 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:59:03.382126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-0949b62f-b604-48e7-a33a-1023bebd3e2f req-a02fb946-c0c4-47db-b28d-f127473ca37e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3cb025a5-578e-4cef-9ba2-3bc0d4e17633", "name": "network-vif-deleted", "tag": "235589b0-b52d-44dc-9b6d-b670d6128608"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:03.392333 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-0949b62f-b604-48e7-a33a-1023bebd3e2f req-a02fb946-c0c4-47db-b28d-f127473ca37e service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:03.392809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-0949b62f-b604-48e7-a33a-1023bebd3e2f req-a02fb946-c0c4-47db-b28d-f127473ca37e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:03.393230 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-0949b62f-b604-48e7-a33a-1023bebd3e2f req-a02fb946-c0c4-47db-b28d-f127473ca37e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:03.405040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-0949b62f-b604-48e7-a33a-1023bebd3e2f req-a02fb946-c0c4-47db-b28d-f127473ca37e service nova] Creating event network-vif-deleted:235589b0-b52d-44dc-9b6d-b670d6128608 for instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 on hv-cinder-83587 Oct 05 14:59:03.410265 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-0949b62f-b604-48e7-a33a-1023bebd3e2f req-a02fb946-c0c4-47db-b28d-f127473ca37e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099198 Oct 05 14:59:03.411237 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 340/637] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:59:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:03.717275 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:03.717996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:03.724205 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:03.724686 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:03.725119 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:03.725830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:03.726276 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:03.726694 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:03.756951 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:03.757355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:03.757821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:03.758260 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lazy-loading 'fault' on Instance uuid 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:59:03.837091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:03.844979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c2b6d5f6-b276-4297-a0ea-2798089d412e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 200 len: 1308 microversion: 2.1 time: 0.132012 Oct 05 14:59:03.845797 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 341/638] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:03 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 1308 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:59:04.017371 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:04.018022 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:04.024579 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:04.025074 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:04.025512 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:04.026258 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:04.026748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:04.027162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:04.046626 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:04.510393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:04.518871 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d6de7449-f56e-42f7-9483-4df8cee888de tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1675 microversion: 2.1 time: 0.505648 Oct 05 14:59:04.519469 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 342/639] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:04 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1675 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:04.872798 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:04.873873 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 3cb025a5-578e-4cef-9ba2-3bc0d4e17633] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:04.909324 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:04.909870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:04.910301 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:04.911015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:04.911456 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:04.911865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:04.940455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 could not be found. Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 HTTP/1.1 Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:04.942348 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 could not be found. Oct 05 14:59:04.943896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '3cb025a5-578e-4cef-9ba2-3bc0d4e17633'} Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 could not be found. Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:04.944756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 could not be found. Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 could not be found. Oct 05 14:59:04.946126 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:04.947494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 3cb025a5-578e-4cef-9ba2-3bc0d4e17633 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:04.947494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5d98b1dc-fe36-44a0-8716-6f44eb121402 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633" status: 404 len: 111 microversion: 2.1 time: 0.077247 Oct 05 14:59:04.947494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 343/640] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:04 2022] GET /compute/v2.1/servers/3cb025a5-578e-4cef-9ba2-3bc0d4e17633 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 05 14:59:04.949405 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-f5173c8b-b4c6-43a2-ad18-5a141125fcef req-13181a39-c586-4132-afe9-1619b873b3aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b0af790-f4eb-48b1-b03e-06f2cb5668c7", "name": "network-vif-deleted", "tag": "48ab4437-5a55-48e1-ade3-623e768c36db"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:04.960597 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f5173c8b-b4c6-43a2-ad18-5a141125fcef req-13181a39-c586-4132-afe9-1619b873b3aa service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:04.962321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f5173c8b-b4c6-43a2-ad18-5a141125fcef req-13181a39-c586-4132-afe9-1619b873b3aa service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:04.962769 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-f5173c8b-b4c6-43a2-ad18-5a141125fcef req-13181a39-c586-4132-afe9-1619b873b3aa service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:04.975774 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-f5173c8b-b4c6-43a2-ad18-5a141125fcef req-13181a39-c586-4132-afe9-1619b873b3aa service nova] Creating event network-vif-deleted:48ab4437-5a55-48e1-ade3-623e768c36db for instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 on hv-cinder-83587 Oct 05 14:59:04.987087 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-f5173c8b-b4c6-43a2-ad18-5a141125fcef req-13181a39-c586-4132-afe9-1619b873b3aa service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038825 Oct 05 14:59:04.987807 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 344/641] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:59:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:05.544065 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:05.545530 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:05.561504 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:05.562207 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:05.562923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:05.565183 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:05.566081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:05.566821 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:05.621415 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:05.622058 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:05.627385 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:05.628103 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'fault' on Instance uuid 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 14:59:05.678244 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:05.689071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-4f5aeade-f09a-4670-9dce-d3d69948ea63 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 200 len: 1328 microversion: 2.1 time: 0.150268 Oct 05 14:59:05.689690 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 345/642] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:05 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 1328 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:59:06.714048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:06.715552 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:06.722649 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:06.723396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:06.724036 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:06.725116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:06.726006 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:06.726591 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:06.742095 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 HTTP/1.1 Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:06.744070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 14:59:06.745863 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7'} Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:06.747169 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 14:59:06.748768 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:06.751695 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:06.753129 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-0b92a47b-8111-4f0e-934c-856442eb3d14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 404 len: 111 microversion: 2.1 time: 0.043168 Oct 05 14:59:06.754584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 346/643] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:06 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:06.762965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ff7a5c96-7c2b-47f0-b0b9-4a8f28122b05 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '.version_select at 0x7f77c96feaf0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:06.768274 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR oslo.messaging._drivers.impl_rabbit [None req-ff7a5c96-7c2b-47f0-b0b9-4a8f28122b05 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [c711a470-43b6-4229-bf86-92e1f4ed2b45] AMQP server on 10.222.0.23:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 05 14:59:07.784078 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [None req-ff7a5c96-7c2b-47f0-b0b9-4a8f28122b05 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [c711a470-43b6-4229-bf86-92e1f4ed2b45] Reconnected to AMQP server on 10.222.0.23:5672 via [amqp] client with port 49160. Oct 05 14:59:07.806270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ff7a5c96-7c2b-47f0-b0b9-4a8f28122b05 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1239015976" status: 202 len: 0 microversion: 2.1 time: 1.047092 Oct 05 14:59:07.807182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 347/644] 10.222.0.23 () {58 vars in 1221 bytes} [Wed Oct 5 14:59:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1239015976 => generated 0 bytes in 1049 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 05 14:59:08.023396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-059aadcc-5e3e-4e6d-89d2-6ec13add5c18 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Action: 'create', calling method: .version_select at 0x7f77c9681a60>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-914610862"}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:08.321797 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-059aadcc-5e3e-4e6d-89d2-6ec13add5c18 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.421986 Oct 05 14:59:08.323155 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 348/645] 10.222.0.23 () {60 vars in 1155 bytes} [Wed Oct 5 14:59:07 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 424 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 05 14:59:08.346984 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-26fa3e89-f736-4186-8ff8-764b1b6d07e3 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c9680160>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:08.402915 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-26fa3e89-f736-4186-8ff8-764b1b6d07e3 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 200 len: 344 microversion: 2.1 time: 0.070670 Oct 05 14:59:08.403562 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 349/646] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:59:08 2022] GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 344 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:08.486103 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6b68c85e-16fa-4d93-b322-3a24fc993adb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'create', calling method: .version_select at 0x7f77c95d1280>, body: {"keypair": {"name": "tempest-keypair-1971162760", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK41a1FmB8gsr4crZmwny5+jHeDvj5SsLOJ+wxojqPrtsEfn8uP5ZwTQNEbWlXicNsrrzbSmielqxXChGAmF4soNpThzagTjYmECbsAh+ZYCtNg9BRfaA5fwBX70O7YoEQ=="}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:08.501551 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-6b68c85e-16fa-4d93-b322-3a24fc993adb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for project ff81348c32db4e40a8e331317d14e3d2. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:59:08.505520 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-6b68c85e-16fa-4d93-b322-3a24fc993adb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for user 7de0d01f891d4d73bc44810aacbca477 and project ff81348c32db4e40a8e331317d14e3d2. Resources: {'key_pairs'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:59:08.529420 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6b68c85e-16fa-4d93-b322-3a24fc993adb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.047313 Oct 05 14:59:08.530045 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 350/647] 10.222.0.23 () {60 vars in 1159 bytes} [Wed Oct 5 14:59:08 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:09.417951 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d5abbcad-8c23-44c0-97f6-a7654e7ccc77 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c96feca0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:09.474866 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d5abbcad-8c23-44c0-97f6-a7654e7ccc77 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 200 len: 344 microversion: 2.1 time: 0.060455 Oct 05 14:59:09.475645 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 351/648] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:59:09 2022] GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 344 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:09.937678 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-399935159", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5c24f205-c58b-44e6-8aed-44447bb6425d"}], "security_groups": [{"name": "tempest-securitygroup--1838766678"}], "key_name": "tempest-keypair-1971162760", "user_data": "}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:10.046683 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.network.neutron [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] validate_networks() for [('5c24f205-c58b-44e6-8aed-44447bb6425d', None, None, None, None, None)] {{(pid=120605) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:59:10.540153 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Flavor limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:59:10.540750 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Image limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:59:10.541045 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Flavor pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:59:10.541442 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Image pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:59:10.541943 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:59:10.551042 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:10.551525 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:10.551950 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:10.552659 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:10.553100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:10.553513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:10.569776 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for project ff81348c32db4e40a8e331317d14e3d2. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:59:10.573613 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Getting quotas for user 7de0d01f891d4d73bc44810aacbca477 and project ff81348c32db4e40a8e331317d14e3d2. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:59:10.584232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Going to run 1 instances... {{(pid=120605) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:59:10.586894 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:59:10.641706 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b0fb4371-f173-4dfb-a8af-c316a67d9f40 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.708171 Oct 05 14:59:10.642649 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 352/649] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:59:09 2022] POST /compute/v2.1/servers => generated 402 bytes in 710 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:59:10.647069 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-d4eb27fc-8901-41f8-b479-77c28246ba11 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c9681dc0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:10.695195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-d4eb27fc-8901-41f8-b479-77c28246ba11 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 200 len: 344 microversion: 2.1 time: 0.051490 Oct 05 14:59:10.695783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 353/650] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:59:10 2022] GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 344 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:10.705699 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1a085959-7c12-4850-b470-bf1d9e4031be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:10.706303 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1a085959-7c12-4850-b470-bf1d9e4031be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:10.759746 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1a085959-7c12-4850-b470-bf1d9e4031be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:10.767792 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1a085959-7c12-4850-b470-bf1d9e4031be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1237 microversion: 2.1 time: 0.070888 Oct 05 14:59:10.768363 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 354/651] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:10 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1237 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:11.710257 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8a22e731-0e60-47ef-9468-42585efb3c61 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c96fec10>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:11.760094 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8a22e731-0e60-47ef-9468-42585efb3c61 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 200 len: 345 microversion: 2.1 time: 0.054271 Oct 05 14:59:11.760847 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 355/652] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:59:11 2022] GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 345 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:11.773355 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b1c51b13-e87c-44b7-8f76-a6724c5732a1 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Action: 'create', calling method: .version_select at 0x7f77c97b5280>, body: {"snapshot": {"volume_id": "868fe3fb-ca2e-493b-92e9-cde8eaf4c916", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1337665940"}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:11.933847 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b1c51b13-e87c-44b7-8f76-a6724c5732a1 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.163976 Oct 05 14:59:11.934554 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 356/653] 10.222.0.23 () {60 vars in 1162 bytes} [Wed Oct 5 14:59:11 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:11.939508 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:11.940813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:11.957340 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:11.957827 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:11.958264 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:11.958990 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:11.959513 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:11.960300 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:12.040193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:12.048903 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ca7ca00f-0213-4c48-8168-b61d012cf087 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1307 microversion: 2.1 time: 0.113027 Oct 05 14:59:12.049463 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 357/654] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:11 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1307 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:12.068365 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6336679c-d659-44db-9c55-d515ac1a7157 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c9681b80>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:12.098397 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6336679c-d659-44db-9c55-d515ac1a7157 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.033750 Oct 05 14:59:12.098907 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 358/655] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:12 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:12.109590 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e09025a0-932f-4eaa-8076-bf3f98286ddf tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c9903430>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:12.139933 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e09025a0-932f-4eaa-8076-bf3f98286ddf tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.033933 Oct 05 14:59:12.140497 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 359/656] 10.222.0.23 () {58 vars in 1140 bytes} [Wed Oct 5 14:59:12 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:12.151364 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2bad6741-6bfe-4daf-9c87-42dfbcdfec0b tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c977e040>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:12.183603 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2bad6741-6bfe-4daf-9c87-42dfbcdfec0b tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.035725 Oct 05 14:59:12.184192 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 360/657] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:12 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:13.072215 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:13.072845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:13.079166 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:13.079689 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:13.080123 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:13.080820 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:13.081275 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:13.081718 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:13.207473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:13.215870 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-59746b23-5b7f-41ac-9703-d3959265f3b1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.147649 Oct 05 14:59:13.216450 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 361/658] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:13 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:13.220793 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-7b96de42-fa05-4a6f-a799-42a6561ca5e6 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c977ea60>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:13.250620 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-7b96de42-fa05-4a6f-a799-42a6561ca5e6 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.033172 Oct 05 14:59:13.251195 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 362/659] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:13 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:14.243134 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:14.244249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:14.254162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:14.254956 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:14.255666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:14.256784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:14.257537 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:14.258250 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:14.396965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:14.408777 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-255a7151-f840-4735-85ad-91f46a9c9438 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.172133 Oct 05 14:59:14.409616 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 363/660] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:14 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:14.416457 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2139082e-2d15-457b-8c8d-1be2f7350512 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c96a5940>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:14.447934 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2139082e-2d15-457b-8c8d-1be2f7350512 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.036974 Oct 05 14:59:14.448465 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 364/661] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:14 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:15.437072 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:15.437985 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:15.446700 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:15.447187 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:15.447826 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:15.448510 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:15.448964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:15.449381 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:15.646844 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:15.657471 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-2c0ebd84-8d7d-4bc8-ba3b-b5f22bafeb14 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.225145 Oct 05 14:59:15.658357 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 365/662] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:15 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:59:15.662957 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-104a22a0-79a2-4ed4-b640-65cdd0da6132 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c97b5940>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:15.693194 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-104a22a0-79a2-4ed4-b640-65cdd0da6132 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.033796 Oct 05 14:59:15.693774 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 366/663] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:15 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:15.793145 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-99d11a5b-bfa0-4e17-a1ad-94f0c4377697 req-3562cd12-32cc-4bc5-b893-75f049df57f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ad3ee7bf-c734-421f-a660-1cc54792201d", "tag": "41672ae1-cc58-4f05-8654-c236d7149e07"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:15.803018 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-99d11a5b-bfa0-4e17-a1ad-94f0c4377697 req-3562cd12-32cc-4bc5-b893-75f049df57f6 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:15.803495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-99d11a5b-bfa0-4e17-a1ad-94f0c4377697 req-3562cd12-32cc-4bc5-b893-75f049df57f6 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:15.803964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-99d11a5b-bfa0-4e17-a1ad-94f0c4377697 req-3562cd12-32cc-4bc5-b893-75f049df57f6 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:15.816769 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-99d11a5b-bfa0-4e17-a1ad-94f0c4377697 req-3562cd12-32cc-4bc5-b893-75f049df57f6 service nova] Creating event network-changed:41672ae1-cc58-4f05-8654-c236d7149e07 for instance ad3ee7bf-c734-421f-a660-1cc54792201d on hv-cinder-83587 Oct 05 14:59:15.822097 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-99d11a5b-bfa0-4e17-a1ad-94f0c4377697 req-3562cd12-32cc-4bc5-b893-75f049df57f6 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032735 Oct 05 14:59:15.822726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 367/664] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:59:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:16.682353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:16.683081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:16.689793 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:16.690250 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:16.690714 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:16.691404 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:16.691850 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:16.692265 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:16.878285 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:16.889797 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-65e5b1be-e6d4-4d1c-87ba-b82e2eaef0bd tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.211483 Oct 05 14:59:16.890643 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 368/665] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:16 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:16.895050 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-eae0099f-be47-4a11-912b-45bfe2981b66 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c97b5550>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:16.931994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-eae0099f-be47-4a11-912b-45bfe2981b66 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.040282 Oct 05 14:59:16.932605 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 369/666] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:16 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:17.917545 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:17.918652 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:17.926109 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:17.926598 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:17.927024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:17.927712 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:17.928160 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:17.928579 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:18.077488 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:18.085896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-67921bac-e753-41bb-88b5-b74f1aff840f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.175002 Oct 05 14:59:18.086481 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 370/667] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:17 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:18.090855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8c089272-1fd3-430c-bc3b-129aebf12bd1 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c96a5ee0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:18.122502 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8c089272-1fd3-430c-bc3b-129aebf12bd1 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 288 microversion: 2.1 time: 0.034976 Oct 05 14:59:18.123076 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 371/668] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:18 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 288 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:18.133830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-f94557c6-c572-47b4-bf4c-e4aa60a77f8d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c97c25e0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:18.193998 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-f94557c6-c572-47b4-bf4c-e4aa60a77f8d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "DELETE /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 202 len: 0 microversion: 2.1 time: 0.063577 Oct 05 14:59:18.195102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 372/669] 10.222.0.23 () {58 vars in 1254 bytes} [Wed Oct 5 14:59:18 2022] DELETE /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 0 bytes in 65 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 14:59:18.202896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-20421c19-a1ab-4867-92f1-4c5216ff3050 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c96fec10>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:18.232410 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-20421c19-a1ab-4867-92f1-4c5216ff3050 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.033081 Oct 05 14:59:18.232986 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 373/670] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:18 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:19.112492 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:19.113682 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:19.118409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:19.124946 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:19.125413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:19.125916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:19.126669 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:19.127037 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:19.127473 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:19.260184 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:19.268147 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-70311d74-3c61-426a-8220-faaa8a5d5c99 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.162416 Oct 05 14:59:19.268715 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 374/671] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:19 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:19.273154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-920dc503-de2c-4f58-9bfb-980cb5d14785 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c9681b80>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:19.303332 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-920dc503-de2c-4f58-9bfb-980cb5d14785 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.033542 Oct 05 14:59:19.303892 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 375/672] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:19 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:20.295181 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:20.296278 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:20.303456 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:20.303992 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:20.304468 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:20.305228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:20.305829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:20.306303 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:20.437441 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:20.445832 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e8d32f48-a24a-41ce-95a4-a3a7074e91be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.157498 Oct 05 14:59:20.446409 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 376/673] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:20 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:20.450776 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-319824eb-9464-477f-8c0e-4600efee2aa7 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c96fe280>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:20.481703 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-319824eb-9464-477f-8c0e-4600efee2aa7 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.034276 Oct 05 14:59:20.482298 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 377/674] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:20 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:21.469349 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:21.470845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=12 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:21.477627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:21.478661 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:21.490222 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:21.491185 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:21.491894 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:21.493013 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:21.493852 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:21.494542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:21.636924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:21.644831 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-85a20520-637f-48f6-abcc-ab7093302f71 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.176897 Oct 05 14:59:21.645408 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 378/675] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:21 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:21.649855 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c7b537aa-7435-4d62-b144-4a6a3bf7ba9d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c9a70af0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:21.680598 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c7b537aa-7435-4d62-b144-4a6a3bf7ba9d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 200 len: 287 microversion: 2.1 time: 0.034097 Oct 05 14:59:21.681132 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 379/676] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:21 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:22.271419 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 14:59:22.271937 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:59:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:59:22.672361 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:22.673429 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:22.684069 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:22.685081 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:22.686303 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:22.688168 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:22.689380 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:22.690212 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:22.830598 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:22.839213 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-88f022aa-ae1d-4e77-9019-267b759db5ba tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.173936 Oct 05 14:59:22.839784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 380/677] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:22 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:22.844149 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-de1db131-b043-45e3-a421-0e63ad66b30b tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c98efc10>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:22.869594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-de1db131-b043-45e3-a421-0e63ad66b30b tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] HTTP exception thrown: Snapshot 86b90c62-b6d2-40cb-87ca-67e617023e93 could not be found. Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-de1db131-b043-45e3-a421-0e63ad66b30b tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Request method failure captured: Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 HTTP/1.1 Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZu75hcdWNjVJ3zbc8tMxyEWeOjQVvnoT3lRnkIQg5PioeAClDBc4qkp3pP0wVRBns5prlXF4XqK8XSevzzHDQlD0pwfRNYbh8OTEeN_krqB9E39CbpGiruS1905spAkmOOv2sYoZ0jNzXVbJ5I38wSuFK4nL_VWs2Jy_90YWQjBzaY Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:22.872015 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_80d855ec2539489a9565cc2892315976", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: d1f3c7f91835440fafbf607c947b9139 Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Snapshot 86b90c62-b6d2-40cb-87ca-67e617023e93 could not be found. Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '86b90c62-b6d2-40cb-87ca-67e617023e93'} Oct 05 14:59:22.873865 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 86b90c62-b6d2-40cb-87ca-67e617023e93 could not be found. (HTTP 404) (Request-ID: req-410582c6-92cd-4b38-9762-381ac7f35746) Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:22.874979 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:59:22.876594 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 86b90c62-b6d2-40cb-87ca-67e617023e93 could not be found. Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 05 14:59:22.878227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 86b90c62-b6d2-40cb-87ca-67e617023e93 could not be found. Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 86b90c62-b6d2-40cb-87ca-67e617023e93 could not be found. Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-de1db131-b043-45e3-a421-0e63ad66b30b tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Returning 404 to user: Snapshot 86b90c62-b6d2-40cb-87ca-67e617023e93 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:22.879845 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-de1db131-b043-45e3-a421-0e63ad66b30b tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93" status: 404 len: 111 microversion: 2.1 time: 0.032725 Oct 05 14:59:22.880920 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 381/678] 10.222.0.23 () {58 vars in 1251 bytes} [Wed Oct 5 14:59:22 2022] GET /compute/v2.1/os-snapshots/86b90c62-b6d2-40cb-87ca-67e617023e93 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:22.882083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-dea51b92-f335-4ce8-98d9-6c9a3be7e000 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c977e670>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:22.951102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-dea51b92-f335-4ce8-98d9-6c9a3be7e000 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 202 len: 0 microversion: 2.1 time: 0.072626 Oct 05 14:59:22.951670 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 382/679] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:59:22 2022] DELETE /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 0 bytes in 74 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 05 14:59:22.960291 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e880155d-d949-44f5-ba8e-a8a8ab6e385a tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77ca50d940>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:23.021495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e880155d-d949-44f5-ba8e-a8a8ab6e385a tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 200 len: 344 microversion: 2.1 time: 0.065367 Oct 05 14:59:23.022127 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 383/680] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:59:22 2022] GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 344 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:23.410895 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1335161982", "imageRef": "1d0a5f0e-e091-424c-bf15-980bcffc3f12", "flavorRef": "452", "networks": [{"uuid": "83ad587d-481a-4572-af34-a5e33a290738"}]}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:23.468330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.network.neutron [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] validate_networks() for [('83ad587d-481a-4572-af34-a5e33a290738', None, None, None, None, None)] {{(pid=120605) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 05 14:59:24.059815 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Flavor limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 05 14:59:24.060500 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Image limits 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 05 14:59:24.060661 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Flavor pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 05 14:59:24.061083 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Image pref 0:0:0 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 05 14:59:24.061476 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.virt.hardware [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120605) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 05 14:59:24.070763 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:24.071236 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:24.071655 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:24.072361 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:24.072782 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:24.073218 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:24.090107 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Getting quotas for project 49adc8da67bf4529a349a2bd79e87e07. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 05 14:59:24.094280 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.quota [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Getting quotas for user bc27904d964e43438c32876b8a1b202d and project 49adc8da67bf4529a349a2bd79e87e07. Resources: {'instances', 'ram', 'cores'} {{(pid=120605) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 05 14:59:24.105588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Going to run 1 instances... {{(pid=120605) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 05 14:59:24.107941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1d0a5f0e-e091-424c-bf15-980bcffc3f12',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120605) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 05 14:59:24.156338 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e142b25e-490b-4706-82a8-5969f8418cfe tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.750518 Oct 05 14:59:24.156997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 384/681] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:59:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 752 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 05 14:59:24.161443 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:24.162071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:24.168598 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:24.169072 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:24.169495 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:24.170208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:24.170654 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:24.171061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:24.302759 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:24.311024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-c53dd02b-94b0-4bc8-9f6d-ea7a0bb58e23 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.152725 Oct 05 14:59:24.319449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 385/682] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:24 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:24.319449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-044ff4d5-22ff-457e-8073-33f7f605f42d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c977e3a0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:24.344333 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-044ff4d5-22ff-457e-8073-33f7f605f42d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] HTTP exception thrown: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-044ff4d5-22ff-457e-8073-33f7f605f42d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Request method failure captured: Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 HTTP/1.1 Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZu75hcdWNjVJ3zbc8tMxyEWeOjQVvnoT3lRnkIQg5PioeAClDBc4qkp3pP0wVRBns5prlXF4XqK8XSevzzHDQlD0pwfRNYbh8OTEeN_krqB9E39CbpGiruS1905spAkmOOv2sYoZ0jNzXVbJ5I38wSuFK4nL_VWs2Jy_90YWQjBzaY Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:24.346317 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_80d855ec2539489a9565cc2892315976", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: d1f3c7f91835440fafbf607c947b9139 Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '868fe3fb-ca2e-493b-92e9-cde8eaf4c916'} Oct 05 14:59:24.348329 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. (HTTP 404) (Request-ID: req-b9a2738c-bc8a-40e8-a983-4be65b0e1699) Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.349413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:59:24.351028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 05 14:59:24.352557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-044ff4d5-22ff-457e-8073-33f7f605f42d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Returning 404 to user: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-044ff4d5-22ff-457e-8073-33f7f605f42d tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 404 len: 109 microversion: 2.1 time: 0.035508 Oct 05 14:59:24.354321 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 386/683] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:59:24 2022] GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:24.355675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b24cbf02-e5ef-4265-85f4-338a0ffdcc0d tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:24.355675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b24cbf02-e5ef-4265-85f4-338a0ffdcc0d tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:24.417531 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b24cbf02-e5ef-4265-85f4-338a0ffdcc0d tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:24.432945 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b24cbf02-e5ef-4265-85f4-338a0ffdcc0d tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1219 microversion: 2.1 time: 0.082747 Oct 05 14:59:24.434489 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 387/684] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:24 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1219 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:24.443048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cf4711ae-c15a-4eeb-9392-ac9c74b1dcaf tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c977ef70>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:24.482229 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-cf4711ae-c15a-4eeb-9392-ac9c74b1dcaf tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] HTTP exception thrown: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cf4711ae-c15a-4eeb-9392-ac9c74b1dcaf tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Request method failure captured: Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 HTTP/1.1 Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZu75hcdWNjVJ3zbc8tMxyEWeOjQVvnoT3lRnkIQg5PioeAClDBc4qkp3pP0wVRBns5prlXF4XqK8XSevzzHDQlD0pwfRNYbh8OTEeN_krqB9E39CbpGiruS1905spAkmOOv2sYoZ0jNzXVbJ5I38wSuFK4nL_VWs2Jy_90YWQjBzaY Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:24.487503 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_80d855ec2539489a9565cc2892315976", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: d1f3c7f91835440fafbf607c947b9139 Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '868fe3fb-ca2e-493b-92e9-cde8eaf4c916'} Oct 05 14:59:24.489539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. (HTTP 404) (Request-ID: req-f9cbb821-66ee-40b0-a7fb-7a5a135acf26) Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.491030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 05 14:59:24.492675 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 05 14:59:24.494228 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cf4711ae-c15a-4eeb-9392-ac9c74b1dcaf tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Returning 404 to user: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-cf4711ae-c15a-4eeb-9392-ac9c74b1dcaf tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "DELETE /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 404 len: 109 microversion: 2.1 time: 0.054042 Oct 05 14:59:24.495739 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 388/685] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 14:59:24 2022] DELETE /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 109 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:24.501356 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-44733009-9d46-4b4d-9439-ffbde395a0d2 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Calling method '.version_select at 0x7f77c96feca0>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:24.533436 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-44733009-9d46-4b4d-9439-ffbde395a0d2 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] HTTP exception thrown: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-44733009-9d46-4b4d-9439-ffbde395a0d2 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Request method failure captured: Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 HTTP/1.1 Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZu75hcdWNjVJ3zbc8tMxyEWeOjQVvnoT3lRnkIQg5PioeAClDBc4qkp3pP0wVRBns5prlXF4XqK8XSevzzHDQlD0pwfRNYbh8OTEeN_krqB9E39CbpGiruS1905spAkmOOv2sYoZ0jNzXVbJ5I38wSuFK4nL_VWs2Jy_90YWQjBzaY Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 14:59:24.536164 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_80d855ec2539489a9565cc2892315976", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/80d855ec2539489a9565cc2892315976"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/80d855ec2539489a9565cc2892315976"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: 80d855ec2539489a9565cc2892315976 Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1466913126 Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: d1f3c7f91835440fafbf607c947b9139 Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1466913126-project Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '868fe3fb-ca2e-493b-92e9-cde8eaf4c916'} Oct 05 14:59:24.537627 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. (HTTP 404) (Request-ID: req-227880fe-0348-428c-ab96-24084710fec5) Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.538581 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 05 14:59:24.539830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 05 14:59:24.541057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-44733009-9d46-4b4d-9439-ffbde395a0d2 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] Returning 404 to user: Volume 868fe3fb-ca2e-493b-92e9-cde8eaf4c916 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-44733009-9d46-4b4d-9439-ffbde395a0d2 tempest-VolumesSnapshotsTestJSON-1466913126 tempest-VolumesSnapshotsTestJSON-1466913126-project] 10.222.0.23 "GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916" status: 404 len: 109 microversion: 2.1 time: 0.041199 Oct 05 14:59:24.542789 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 389/686] 10.222.0.23 () {58 vars in 1245 bytes} [Wed Oct 5 14:59:24 2022] GET /compute/v2.1/os-volumes/868fe3fb-ca2e-493b-92e9-cde8eaf4c916 => generated 109 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 14:59:25.333774 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:25.334551 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:25.342576 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:25.343408 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:25.344116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:25.345269 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:25.345909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:25.346584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:25.544764 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:25.553064 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-eb2dd697-c1db-4ce8-b062-84e906648cc6 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.223549 Oct 05 14:59:25.569101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 390/687] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:25 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:59:25.569101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:25.569101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:25.569101 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:25.569646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:25.569646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:25.569646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:25.569646 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:25.570030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:25.644575 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:25.648426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:25.653909 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-bfb67e3e-c82c-4881-8a79-9388208af674 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1289 microversion: 2.1 time: 0.098952 Oct 05 14:59:25.654998 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 391/688] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:25 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1289 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:26.578771 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:26.580270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:26.588527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:26.589033 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:26.589599 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:26.590330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:26.590895 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:26.591324 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:26.805527 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:26.814809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-bf0b7481-b774-4ba8-8a11-1a47a76c1d65 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.241630 Oct 05 14:59:26.815688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 392/689] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:26 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 05 14:59:26.968385 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-8da680a9-a76e-4741-8d3d-a4da45b2410a req-ffcfd578-2957-4206-9fe7-b96f3983914e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad3ee7bf-c734-421f-a660-1cc54792201d", "name": "network-vif-plugged", "status": "completed", "tag": "41672ae1-cc58-4f05-8654-c236d7149e07"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:26.979367 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-8da680a9-a76e-4741-8d3d-a4da45b2410a req-ffcfd578-2957-4206-9fe7-b96f3983914e service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:26.980106 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-8da680a9-a76e-4741-8d3d-a4da45b2410a req-ffcfd578-2957-4206-9fe7-b96f3983914e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:26.980509 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-8da680a9-a76e-4741-8d3d-a4da45b2410a req-ffcfd578-2957-4206-9fe7-b96f3983914e service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:26.995496 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-8da680a9-a76e-4741-8d3d-a4da45b2410a req-ffcfd578-2957-4206-9fe7-b96f3983914e service nova] Creating event network-vif-plugged:41672ae1-cc58-4f05-8654-c236d7149e07 for instance ad3ee7bf-c734-421f-a660-1cc54792201d on hv-cinder-83587 Oct 05 14:59:27.011136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-8da680a9-a76e-4741-8d3d-a4da45b2410a req-ffcfd578-2957-4206-9fe7-b96f3983914e service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.193238 Oct 05 14:59:27.012886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 393/690] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:59:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:27.018452 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:27.019379 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:27.028028 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:27.029048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:27.029846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:27.030740 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:27.031420 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:27.032116 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:27.273114 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:27.281481 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-cfc29886-8fe8-4e6d-9bfa-3b37281670d9 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.267494 Oct 05 14:59:27.293854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 394/691] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:27 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:27.293854 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-414bea70-3ba1-4164-954b-180f360597c7 req-2fd63147-70a9-4852-ac9c-a74f8e828759 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "85f6c458-4088-4b88-a746-5179b8c43b36", "tag": "1d1cd6fa-e8f9-45a8-b053-27e5bbfbd93c"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:27.296645 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-414bea70-3ba1-4164-954b-180f360597c7 req-2fd63147-70a9-4852-ac9c-a74f8e828759 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:27.297359 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-414bea70-3ba1-4164-954b-180f360597c7 req-2fd63147-70a9-4852-ac9c-a74f8e828759 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:27.298026 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-414bea70-3ba1-4164-954b-180f360597c7 req-2fd63147-70a9-4852-ac9c-a74f8e828759 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:27.310102 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-414bea70-3ba1-4164-954b-180f360597c7 req-2fd63147-70a9-4852-ac9c-a74f8e828759 service nova] Creating event network-changed:1d1cd6fa-e8f9-45a8-b053-27e5bbfbd93c for instance 85f6c458-4088-4b88-a746-5179b8c43b36 on hv-cinder-83587 Oct 05 14:59:27.315924 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-414bea70-3ba1-4164-954b-180f360597c7 req-2fd63147-70a9-4852-ac9c-a74f8e828759 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031876 Oct 05 14:59:27.316595 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 395/692] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:59:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:27.842338 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:27.843096 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:27.850966 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:27.851454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:27.851886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:27.852577 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:27.853030 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:27.853451 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:27.986996 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:27.998421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-830661e0-1a89-43a1-8790-8a6575d6d662 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.161833 Oct 05 14:59:27.999210 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 396/693] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:27 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:28.306196 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:28.306810 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:28.313100 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:28.313580 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:28.314035 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:28.314718 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:28.315162 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:28.315571 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:28.538413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:28.546654 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ade83ea0-c5ce-4de4-a068-f1bbe2070f15 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.244451 Oct 05 14:59:28.547330 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 397/694] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:28 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:29.024625 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:29.025449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:29.033528 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:29.034168 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:29.034700 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:29.035569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:29.036159 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:29.036648 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:29.167813 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:29.175426 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-e7a43af2-6ac3-48ee-ad47-243548211d9f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.156035 Oct 05 14:59:29.176049 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 398/695] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:29 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:29.570519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:29.570519 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:29.576749 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:29.577233 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:29.577788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:29.578367 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:29.578809 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:29.579223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:29.725556 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:29.734861 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-6a5e9859-1a56-46ae-99b7-c994a402f113 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.169094 Oct 05 14:59:29.737090 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 399/696] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:29 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:30.201965 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:30.203052 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:30.213378 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:30.214225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:30.214935 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:30.216057 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:30.216793 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:30.217479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:30.351480 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:30.359903 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-32189ca4-9048-4c48-9f42-f18d65b2b2b4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1363 microversion: 2.1 time: 0.164212 Oct 05 14:59:30.360552 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 400/697] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:30 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1363 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:30.757982 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:30.758564 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:30.764714 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:30.765146 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:30.765572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:30.766281 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:30.766726 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:30.767141 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:30.903826 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:30.912001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-fcbfbfac-8b0d-42f1-ae14-a9334de2bd64 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.157983 Oct 05 14:59:30.912702 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 401/698] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:30 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:31.387976 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:31.389069 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:31.399217 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:31.400140 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:31.400899 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:31.402115 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:31.402997 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:31.403719 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:31.641717 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:31.649714 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-200383c3-e8c6-47d0-acc0-ad206a3797d2 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1550 microversion: 2.1 time: 0.268456 Oct 05 14:59:31.650280 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 402/699] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:31 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1550 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:31.672567 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:31.673065 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:31.679374 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:31.679864 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:31.680298 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:31.680980 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:31.681438 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:31.681884 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:31.771075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a881acf2-86fe-4b3d-88c3-700ac96a4056 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.102409 Oct 05 14:59:31.771688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 403/700] 10.222.0.23 () {58 vars in 1275 bytes} [Wed Oct 5 14:59:31 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-interface => generated 285 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:31.935964 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:31.936574 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:31.942869 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:31.943360 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:31.943788 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:31.944494 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:31.944923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:31.945337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:32.240838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:32.248932 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5d291a8b-1839-4dfe-a563-df7a217751ff tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.316937 Oct 05 14:59:32.249479 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 404/701] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:31 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:33.201174 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-7ca03918-8ddf-45b6-89f5-c8301cb4195c req-d4d5b4d4-7afb-4aaf-9223-cb08bc493b34 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ad3ee7bf-c734-421f-a660-1cc54792201d", "tag": "41672ae1-cc58-4f05-8654-c236d7149e07"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:33.210916 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-7ca03918-8ddf-45b6-89f5-c8301cb4195c req-d4d5b4d4-7afb-4aaf-9223-cb08bc493b34 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:33.211391 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-7ca03918-8ddf-45b6-89f5-c8301cb4195c req-d4d5b4d4-7afb-4aaf-9223-cb08bc493b34 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:33.211812 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-7ca03918-8ddf-45b6-89f5-c8301cb4195c req-d4d5b4d4-7afb-4aaf-9223-cb08bc493b34 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:33.223154 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-7ca03918-8ddf-45b6-89f5-c8301cb4195c req-d4d5b4d4-7afb-4aaf-9223-cb08bc493b34 service nova] Creating event network-changed:41672ae1-cc58-4f05-8654-c236d7149e07 for instance ad3ee7bf-c734-421f-a660-1cc54792201d on hv-cinder-83587 Oct 05 14:59:33.228772 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-7ca03918-8ddf-45b6-89f5-c8301cb4195c req-d4d5b4d4-7afb-4aaf-9223-cb08bc493b34 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030980 Oct 05 14:59:33.229421 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 405/702] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:59:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 05 14:59:33.272499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:33.273253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:33.279661 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:33.280146 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:33.280570 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:33.282001 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:33.282454 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:33.288017 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:33.510787 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:33.526327 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-78221c1b-4e3b-4b76-9d84-c629850221a7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.254823 Oct 05 14:59:33.527573 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 406/703] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:33 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:34.549302 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:34.549947 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:34.553070 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:34.553783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell0:SELECT=8 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:34.557736 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:34.558222 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:34.558653 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:34.559337 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:34.559816 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:34.560229 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:34.691896 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:34.699838 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-03f5659b-e6cd-4a8a-86fe-3eb5c26702e7 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.154284 Oct 05 14:59:34.700542 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 407/704] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:34 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:34.705253 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-3d83404d-1968-4ebc-b774-219ecb52d561 req-3c6cd1d6-d38a-45dd-bbfb-6b9bfa3c3bbf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85f6c458-4088-4b88-a746-5179b8c43b36", "name": "network-vif-plugged", "status": "completed", "tag": "1d1cd6fa-e8f9-45a8-b053-27e5bbfbd93c"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:34.715208 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-3d83404d-1968-4ebc-b774-219ecb52d561 req-3c6cd1d6-d38a-45dd-bbfb-6b9bfa3c3bbf service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:34.715762 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-3d83404d-1968-4ebc-b774-219ecb52d561 req-3c6cd1d6-d38a-45dd-bbfb-6b9bfa3c3bbf service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:34.716249 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-3d83404d-1968-4ebc-b774-219ecb52d561 req-3c6cd1d6-d38a-45dd-bbfb-6b9bfa3c3bbf service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:34.728297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-3d83404d-1968-4ebc-b774-219ecb52d561 req-3c6cd1d6-d38a-45dd-bbfb-6b9bfa3c3bbf service nova] Creating event network-vif-plugged:1d1cd6fa-e8f9-45a8-b053-27e5bbfbd93c for instance 85f6c458-4088-4b88-a746-5179b8c43b36 on hv-cinder-83587 Oct 05 14:59:34.733040 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-3d83404d-1968-4ebc-b774-219ecb52d561 req-3c6cd1d6-d38a-45dd-bbfb-6b9bfa3c3bbf service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031346 Oct 05 14:59:34.733624 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 408/705] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 14:59:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:35.728368 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:35.729351 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:35.739674 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:35.740452 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:35.741174 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:35.742334 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:35.743024 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:35.743692 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:35.879629 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:35.887562 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9ea50ba3-3b52-4708-afbc-323994b38a8e tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1319 microversion: 2.1 time: 0.166500 Oct 05 14:59:35.888136 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 409/706] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:35 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1319 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:36.910297 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 14:59:36.910953 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:36.917475 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:36.918008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:36.918441 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:36.919137 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:36.919583 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:36.920003 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:37.049977 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 14:59:37.059462 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-5025b750-92d2-4a7e-a87a-78488ec5f36c tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1507 microversion: 2.1 time: 0.153183 Oct 05 14:59:37.060035 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 410/707] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 14:59:36 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1507 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 14:59:37.087559 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "adcaf666-4341-4502-89cc-7ffd107a02ff", "device": "/dev/sdb"}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 14:59:37.090631 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 14:59:37.096689 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:37.097170 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:37.097596 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:37.098309 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 14:59:37.098753 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 14:59:37.099167 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 14:59:37.117220 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=100 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:37.392368 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8fd09bf2-a54a-46fb-897e-a6e1c864c1f6 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "POST /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.308705 Oct 05 14:59:37.392951 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 411/708] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 5 14:59:37 2022] POST /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36/os-volume_attachments => generated 194 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 14:59:40.641827 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG neutronclient.v2_0.client [None req-aeb87f64-fce1-4e02-b70e-5031fed6e6a1 tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Error message: {"error": {"code": 401, "title": "Unauthorized", "message": "The request you have made requires authentication."}} {{(pid=120606) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 05 14:59:40.642636 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-aeb87f64-fce1-4e02-b70e-5031fed6e6a1 tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] 10.222.0.23 "POST /compute/v2.1/servers" status: 500 len: 0 microversion: 2.1 time: 899.858223 Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack [None req-aeb87f64-fce1-4e02-b70e-5031fed6e6a1 tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Caught error: Not authorized.: nova.exception.Unauthorized: Not authorized. Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Traceback (most recent call last): Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 476, in find_resource Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.find_resource_by_id(resource, name_or_id, Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 440, in find_resource_by_id Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack raise exceptions.NotFound(message=not_found_message) Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack neutronclient.common.exceptions.NotFound: Unable to find security_group with id 'tempest-secgroup-smoke-503463352' Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack During handling of the above exception, another exception occurred: Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Traceback (most recent call last): Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack exception_handler_v20(status_code, error_body) Oct 05 14:59:40.651314 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack raise client_exc(message=error_message, Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack neutronclient.common.exceptions.Unauthorized: 401-{'error': {'code': 401, 'title': 'Unauthorized', 'message': 'The request you have made requires authentication.'}} Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Neutron server returns request_ids: ['req-0672f78a-3f29-4034-8c17-78a12bdd3584'] Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack During handling of the above exception, another exception occurred: Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Traceback (most recent call last): Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/__init__.py", line 95, in __call__ Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return req.get_response(self.application) Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 104, in __call__ Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack self._log_req(req, res, start) Oct 05 14:59:40.652662 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack self.force_reraise() Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack raise self.value Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 99, in __call__ Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack res = req.get_response(self.application) Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/osprofiler/web.py", line 111, in __call__ Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return request.get_response(self.application) Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:59:40.654062 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack response = req.get_response(self._app) Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/routes/middleware.py", line 153, in __call__ Oct 05 14:59:40.655433 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack response = self.app(environ, start_response) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return resp(environ, start_response) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 486, in __call__ Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self._process_stack(request, action, action_args, Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack action_result = self.dispatch(meth, request, action_args) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return method(req=request, **action_args) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return f(*args, **kwargs) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 05 14:59:40.656623 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack [Previous line repeated 10 more times] Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack instances, resv_id = self.compute_api.create( Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self._create_instance( Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ) = self._validate_and_build_base_options( Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1074, in _validate_and_build_base_options Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack security_groups = self._check_requested_secgroups( Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 521, in _check_requested_secgroups Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack secgroup_uuid = security_group_api.validate_name(context, secgroup) Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/security_group_api.py", line 68, in validate_name Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return neutronv20.find_resourceid_by_name_or_id( Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 56, in find_resourceid_by_name_or_id Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return find_resource_by_name_or_id(client, resource, name_or_id, Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 49, in find_resource_by_name_or_id Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return client.find_resource(resource, name_or_id, project_id, Oct 05 14:59:40.658013 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 480, in find_resource Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self._find_resource_by_name( Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 457, in _find_resource_by_name Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack data = obj_lister(**params) Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1063, in list_security_groups Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.list('security_groups', self.security_groups_path, Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 372, in list Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack for r in self._pagination(collection, path, **params): Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 387, in _pagination Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack res = self.get(path, params=params) Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.659265 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.retry_request("GET", action, body=body, Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack return self.do_request(method, action, body=body, Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack self._handle_fault_response(status_code, replybody, resp) Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 204, in wrapper Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack raise exception.Unauthorized() Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack nova.exception.Unauthorized: Not authorized. Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack Oct 05 14:59:40.660718 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack [None req-aeb87f64-fce1-4e02-b70e-5031fed6e6a1 tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] http://10.222.0.23/compute/v2.1/servers returned with HTTP 401 Oct 05 14:59:40.662428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-aeb87f64-fce1-4e02-b70e-5031fed6e6a1 tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Returning 401 to user: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 14:59:40.662428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:59:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.23) !!! Oct 05 14:59:40.662428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 14:59:40 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/servers (10.222.0.23) Oct 05 14:59:40.662428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: CRITICAL nova [None req-aeb87f64-fce1-4e02-b70e-5031fed6e6a1 tempest-TestVolumeBootPattern-274852972 tempest-TestVolumeBootPattern-274852972-project] Unhandled error: OSError: write error Oct 05 14:59:40.662428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova OSError: write error Oct 05 14:59:40.662428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova Oct 05 14:59:40.662428 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 298/709] 10.222.0.23 () {60 vars in 1147 bytes} [Wed Oct 5 14:44:40 2022] POST /compute/v2.1/servers => generated 0 bytes in 899870 msecs (HTTP/1.1 401) 9 headers in 0 bytes (1 switches on core 0) Oct 05 15:00:08.232991 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:00:08.233802 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=7,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:00:08.238631 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:00:08.239198 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:00:08.246889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:08.247584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:08.247791 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:08.248596 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:08.249043 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:08.249459 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:08.339886 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lazy-loading 'services' on Instance uuid 85f6c458-4088-4b88-a746-5179b8c43b36 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 15:00:08.418881 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1e5a213d-a56e-4fa5-9703-294350b4a634 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36/os-volume_attachments/adcaf666-4341-4502-89cc-7ffd107a02ff" status: 202 len: 0 microversion: 2.1 time: 0.186945 Oct 05 15:00:08.419455 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 412/710] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 15:00:08 2022] DELETE /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36/os-volume_attachments/adcaf666-4341-4502-89cc-7ffd107a02ff => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 15:00:10.629065 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:00:10.629603 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:00:10.636162 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:10.636671 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:10.637095 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:10.637894 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:10.638369 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:10.638765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:10.665168 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Going to try to terminate instance {{(pid=120606) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 15:00:10.750128 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-8b2e7cf9-9c52-4eec-b422-106469fb93cc tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "DELETE /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 204 len: 0 microversion: 2.1 time: 0.125856 Oct 05 15:00:10.750819 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 299/711] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 15:00:10 2022] DELETE /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 15:00:10.765182 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:00:10.766003 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:00:10.775048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:10.775564 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:10.776059 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:10.776761 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:10.777214 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:10.777673 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:10.909353 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:00:10.917218 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-1c1f370e-48f5-4ca7-b88a-ba2aa4835fa0 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1513 microversion: 2.1 time: 0.159023 Oct 05 15:00:10.917829 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 413/712] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:00:10 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1513 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:00:11.945521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:00:11.946766 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:00:11.957418 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:11.958271 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:11.958971 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:11.960092 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:11.961183 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:11.961923 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:12.096063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:00:12.104504 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-a5add660-9b9e-4f34-b5ed-d6dd4d1dbc77 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1513 microversion: 2.1 time: 0.166327 Oct 05 15:00:12.105094 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 300/713] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:00:11 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1513 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:00:13.131120 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:00:13.132129 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:00:13.140151 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:13.140638 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:13.141062 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:13.141777 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:13.142227 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:13.142640 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:13.269885 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:00:13.277584 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-967fb9da-98e1-4101-af22-a16999c4e38b tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1513 microversion: 2.1 time: 0.153033 Oct 05 15:00:13.278189 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 414/714] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:00:13 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1513 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:00:13.997858 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [req-2f60dd8b-61a1-4c6f-aaba-18c4b2feda08 req-267cadec-4ade-4112-a38b-ebdd4770e956 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85f6c458-4088-4b88-a746-5179b8c43b36", "name": "network-vif-deleted", "tag": "1d1cd6fa-e8f9-45a8-b053-27e5bbfbd93c"}]} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 15:00:14.009875 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-2f60dd8b-61a1-4c6f-aaba-18c4b2feda08 req-267cadec-4ade-4112-a38b-ebdd4770e956 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:14.010467 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-2f60dd8b-61a1-4c6f-aaba-18c4b2feda08 req-267cadec-4ade-4112-a38b-ebdd4770e956 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:14.010892 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [req-2f60dd8b-61a1-4c6f-aaba-18c4b2feda08 req-267cadec-4ade-4112-a38b-ebdd4770e956 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:14.026369 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.compute.server_external_events [req-2f60dd8b-61a1-4c6f-aaba-18c4b2feda08 req-267cadec-4ade-4112-a38b-ebdd4770e956 service nova] Creating event network-vif-deleted:1d1cd6fa-e8f9-45a8-b053-27e5bbfbd93c for instance 85f6c458-4088-4b88-a746-5179b8c43b36 on hv-cinder-83587 Oct 05 15:00:14.033559 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [req-2f60dd8b-61a1-4c6f-aaba-18c4b2feda08 req-267cadec-4ade-4112-a38b-ebdd4770e956 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039818 Oct 05 15:00:14.034351 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 301/715] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 15:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:00:14.300694 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:00:14.301247 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:00:14.308268 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:14.308752 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:14.309173 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:14.309893 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:14.310336 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:14.310744 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:14.343876 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:14.344349 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:14.344773 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:14.345217 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.objects.instance [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lazy-loading 'fault' on Instance uuid 85f6c458-4088-4b88-a746-5179b8c43b36 {{(pid=120605) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 15:00:14.387774 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc27904d964e43438c32876b8a1b202d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49adc8da67bf4529a349a2bd79e87e07', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:00:14.395827 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-b9985615-fa25-4817-bbb7-5d7dc6d83ba3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 200 len: 1309 microversion: 2.1 time: 0.099157 Oct 05 15:00:14.396420 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 415/716] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:00:14 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 1309 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:00:15.424715 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:00:15.425893 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] [instance: 85f6c458-4088-4b88-a746-5179b8c43b36] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:00:15.436427 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:15.437212 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:15.437969 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:15.439087 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:00:15.439823 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:00:15.440509 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:00:15.457382 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] HTTP exception thrown: Instance 85f6c458-4088-4b88-a746-5179b8c43b36 could not be found. Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Request method failure captured: Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 HTTP/1.1 Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZgLusJvJKgK7A4BNRZGZBqFkAscS7YyZMcZDcj_toBcfQQL6n2mEGsKdSt_ANzFmbEXw12qwpheI4wIJukp7SReELEs1wyGi2gOXZiS_FezXoxIbPCGrZxfg-W7ser8U-gtEMzk0lBGV0EJYgUbILSy_eo4Ch11wSXDqGpYrxAHp7I Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 15:00:15.459048 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_49adc8da67bf4529a349a2bd79e87e07", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/49adc8da67bf4529a349a2bd79e87e07"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: 49adc8da67bf4529a349a2bd79e87e07 Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-33480543 Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: bc27904d964e43438c32876b8a1b202d Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-VolumesSnapshotTestJSON-33480543-project Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 85f6c458-4088-4b88-a746-5179b8c43b36 could not be found. Oct 05 15:00:15.460768 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '85f6c458-4088-4b88-a746-5179b8c43b36'} Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85f6c458-4088-4b88-a746-5179b8c43b36 could not be found. Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 15:00:15.462156 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85f6c458-4088-4b88-a746-5179b8c43b36 could not be found. Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85f6c458-4088-4b88-a746-5179b8c43b36 could not be found. Oct 05 15:00:15.463802 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:00:15.465414 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] Returning 404 to user: Instance 85f6c458-4088-4b88-a746-5179b8c43b36 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 15:00:15.465414 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3eaad5fc-051f-4350-aa1d-bcab84cfcbb3 tempest-VolumesSnapshotTestJSON-33480543 tempest-VolumesSnapshotTestJSON-33480543-project] 10.222.0.23 "GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36" status: 404 len: 111 microversion: 2.1 time: 0.043790 Oct 05 15:00:15.465414 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 302/717] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:00:15 2022] GET /compute/v2.1/servers/85f6c458-4088-4b88-a746-5179b8c43b36 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 15:01:52.628290 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=10 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:52.629036 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=29,nova_cell1:INSERT=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:52.630748 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Oct 05 15:01:52.631688 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 05 15:01:52.631852 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Wed Oct 5 15:01:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/ips (ip 10.222.0.23) !!! Oct 05 15:01:52.879805 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:01:52.880160 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:01:52.886569 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:01:52.887075 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:01:52.887500 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:01:52.888180 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:01:52.888641 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:01:52.889034 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:01:52.914991 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8b553acd-df0a-40d4-88dd-049fdaee622e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/ips" status: 200 len: 147 microversion: 2.1 time: 0.290324 Oct 05 15:01:52.915557 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 416/718] 10.222.0.23 () {58 vars in 1248 bytes} [Wed Oct 5 15:01:52 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/ips => generated 147 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:01:54.260706 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:54.261447 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:54.262785 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 05 15:01:54.262957 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Wed Oct 5 15:01:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments (ip 10.222.0.23) !!! Oct 05 15:01:54.268002 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "faa9298a-1a93-4308-8c1f-acf75f83db02"}} {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 15:01:54.270358 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:01:54.279397 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:01:54.279847 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:01:54.280274 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:01:54.280963 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:01:54.281407 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:01:54.281849 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:01:54.589914 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f15ef296-a7a0-4342-84ea-513975f5f73e tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.330801 Oct 05 15:01:54.590521 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 303/719] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 5 15:01:54 2022] POST /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments => generated 194 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:02.205819 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:02.208923 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:02.216232 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:02.216714 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:02.217142 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:02.217889 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:02.218333 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:02.218742 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:02.247540 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-3cde73f2-8c89-45a1-85f6-28888fc4d029 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048446 Oct 05 15:02:02.248087 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 417/720] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 5 15:02:02 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:02.260256 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:02.260814 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:02.267811 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:02.268293 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:02.268721 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:02.269411 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:02.269895 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:02.270362 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:02.303724 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-1e530b84-a9ca-43d8-932c-7aad415a021a tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/faa9298a-1a93-4308-8c1f-acf75f83db02" status: 200 len: 194 microversion: 2.1 time: 0.048343 Oct 05 15:02:02.304275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 304/721] 10.222.0.23 () {58 vars in 1413 bytes} [Wed Oct 5 15:02:02 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/faa9298a-1a93-4308-8c1f-acf75f83db02 => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:03.673225 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7ffb2e43-26b9-402c-9811-c70626c60ecc"}} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 15:02:03.675585 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:03.682730 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:03.683221 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:03.683654 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:03.684382 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:03.684830 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:03.685254 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:03.969666 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-acd66526-9c62-4908-b280-3904e3c35145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "POST /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.300339 Oct 05 15:02:03.970313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 418/722] 10.222.0.23 () {60 vars in 1323 bytes} [Wed Oct 5 15:02:03 2022] POST /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments => generated 194 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:11.607123 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:11.609402 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:11.616699 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:11.617230 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:11.617616 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:11.618483 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:11.618967 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:11.619385 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:11.651440 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-e712e41a-49e5-4370-aeef-0ba5a7df7f88 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.048164 Oct 05 15:02:11.652086 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 305/723] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 5 15:02:11 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments => generated 371 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:11.797313 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:11.797888 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:11.804730 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:11.805206 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:11.805648 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:11.806345 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:11.806787 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:11.807201 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:11.836571 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ff4c1c20-4330-4158-a5a9-f98b98d957da tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/faa9298a-1a93-4308-8c1f-acf75f83db02" status: 200 len: 194 microversion: 2.1 time: 0.044075 Oct 05 15:02:11.837147 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 419/724] 10.222.0.23 () {58 vars in 1413 bytes} [Wed Oct 5 15:02:11 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/faa9298a-1a93-4308-8c1f-acf75f83db02 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:11.854038 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:11.854658 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:11.863948 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:11.864610 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:11.865303 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:11.866036 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:11.866480 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:11.866888 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:12.006065 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.objects.instance [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'services' on Instance uuid ad3ee7bf-c734-421f-a660-1cc54792201d {{(pid=120606) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 15:02:12.144767 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-c2deadaa-cae0-460a-8367-e71b6f27cbae tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/faa9298a-1a93-4308-8c1f-acf75f83db02" status: 202 len: 0 microversion: 2.1 time: 0.297283 Oct 05 15:02:12.145379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 306/725] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 15:02:11 2022] DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/faa9298a-1a93-4308-8c1f-acf75f83db02 => generated 0 bytes in 299 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 15:02:13.273470 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:13.274076 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:13.280784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:13.281287 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:13.281783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:13.282575 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:13.283061 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:13.283499 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:13.314756 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-47a0964b-1823-4c7c-8715-82be5f28fec1 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/7ffb2e43-26b9-402c-9811-c70626c60ecc" status: 200 len: 194 microversion: 2.1 time: 0.046575 Oct 05 15:02:13.315396 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 420/726] 10.222.0.23 () {58 vars in 1413 bytes} [Wed Oct 5 15:02:13 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/7ffb2e43-26b9-402c-9811-c70626c60ecc => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:13.327242 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:13.327794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:13.334259 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:13.334728 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:13.335157 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:13.335843 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:13.336285 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:13.336702 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:13.430003 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.objects.instance [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'services' on Instance uuid ad3ee7bf-c734-421f-a660-1cc54792201d {{(pid=120606) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 15:02:13.511652 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-56fe5b6c-0077-4788-8100-13d78c09ee6f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/7ffb2e43-26b9-402c-9811-c70626c60ecc" status: 202 len: 0 microversion: 2.1 time: 0.188622 Oct 05 15:02:13.512307 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 307/727] 10.222.0.23 () {58 vars in 1416 bytes} [Wed Oct 5 15:02:13 2022] DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments/7ffb2e43-26b9-402c-9811-c70626c60ecc => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 15:02:14.711184 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:14.714898 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:14.724548 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:14.725291 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:14.725994 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:14.727089 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:14.727783 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:14.728423 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:14.764268 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-8fbe6bcc-8f92-4df5-a925-13f7ea10b1be tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.060236 Oct 05 15:02:14.764857 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 421/728] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 5 15:02:14 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments => generated 25 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 05 15:02:14.901413 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:14.903780 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:14.910471 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:14.910955 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:14.911387 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:14.912097 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:14.912549 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:14.912966 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:14.943214 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3e5861d5-0d08-4a4b-aecc-2e80f9e66b1c tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046616 Oct 05 15:02:14.943685 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 308/729] 10.222.0.23 () {58 vars in 1302 bytes} [Wed Oct 5 15:02:14 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 05 15:02:15.006950 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:15.007668 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:15.016864 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:15.017695 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:15.018461 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:15.019474 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:15.019936 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:15.020362 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:15.048375 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Going to try to terminate instance {{(pid=120605) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 05 15:02:15.134223 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-80335fba-c42f-4ea1-93f6-3a7e93c71341 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 204 len: 0 microversion: 2.1 time: 0.130869 Oct 05 15:02:15.134853 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 422/730] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 15:02:15 2022] DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 05 15:02:15.143290 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:15.143901 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:15.150046 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:15.150536 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:15.150970 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:15.151663 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:15.152174 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:15.152572 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:15.291941 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:02:15.304149 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-67194c52-b8e1-4cfd-a964-3543a63dfbad tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1675 microversion: 2.1 time: 0.163553 Oct 05 15:02:15.305053 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 309/731] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:02:15 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1675 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:02:16.337201 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:16.338472 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:16.346828 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:16.347425 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:16.347951 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:16.348791 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:16.349346 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:16.349907 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:16.515389 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.policy [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120605) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:02:16.523778 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-88df5536-496c-48be-8879-3aea1da7c806 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1675 microversion: 2.1 time: 0.193336 Oct 05 15:02:16.524354 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 423/732] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:02:16 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1675 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:02:17.547376 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:17.547938 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:17.554950 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:17.555394 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:17.555847 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:17.556532 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:17.556967 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:17.557379 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:17.687387 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:02:17.695021 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-3741c55e-0095-496c-b63b-3453ee4111cb tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1675 microversion: 2.1 time: 0.151692 Oct 05 15:02:17.695620 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 310/733] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:02:17 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1675 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:02:18.324795 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [req-af3a5d7c-115c-4fc4-bb18-1fa85cee5b51 req-f05fbcb6-2f35-48d1-8541-a28c728f7847 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ad3ee7bf-c734-421f-a660-1cc54792201d", "name": "network-vif-deleted", "tag": "41672ae1-cc58-4f05-8654-c236d7149e07"}]} {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 05 15:02:18.337008 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-af3a5d7c-115c-4fc4-bb18-1fa85cee5b51 req-f05fbcb6-2f35-48d1-8541-a28c728f7847 service nova] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:18.337477 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-af3a5d7c-115c-4fc4-bb18-1fa85cee5b51 req-f05fbcb6-2f35-48d1-8541-a28c728f7847 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:18.338287 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [req-af3a5d7c-115c-4fc4-bb18-1fa85cee5b51 req-f05fbcb6-2f35-48d1-8541-a28c728f7847 service nova] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:18.351416 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.compute.server_external_events [req-af3a5d7c-115c-4fc4-bb18-1fa85cee5b51 req-f05fbcb6-2f35-48d1-8541-a28c728f7847 service nova] Creating event network-vif-deleted:41672ae1-cc58-4f05-8654-c236d7149e07 for instance ad3ee7bf-c734-421f-a660-1cc54792201d on hv-cinder-83587 Oct 05 15:02:18.357990 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [req-af3a5d7c-115c-4fc4-bb18-1fa85cee5b51 req-f05fbcb6-2f35-48d1-8541-a28c728f7847 service nova] 10.222.0.23 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036831 Oct 05 15:02:18.358943 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 424/734] 10.222.0.23 () {64 vars in 1317 bytes} [Wed Oct 5 15:02:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 05 15:02:18.721172 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:18.721980 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:18.729245 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:18.729775 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:18.730206 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:18.730902 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:18.731349 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:18.731765 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:18.771399 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:18.771788 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:18.772205 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:18.772642 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.objects.instance [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lazy-loading 'fault' on Instance uuid ad3ee7bf-c734-421f-a660-1cc54792201d {{(pid=120606) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 05 15:02:18.817204 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.policy [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7de0d01f891d4d73bc44810aacbca477', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ff81348c32db4e40a8e331317d14e3d2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120606) authorize /opt/stack/nova/nova/policy.py:199}} Oct 05 15:02:18.825996 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-f3ef5f49-99f6-42bb-a650-a64da886b603 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 200 len: 1327 microversion: 2.1 time: 0.110691 Oct 05 15:02:18.826606 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 311/735] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:02:18 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 1327 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 05 15:02:19.855577 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:19.856784 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:19.868072 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:19.868553 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:19.868973 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:19.869679 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:19.870123 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:19.870539 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:19.885193 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d HTTP/1.1 Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 15:02:19.886902 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:19.888685 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': 'ad3ee7bf-c734-421f-a660-1cc54792201d'} Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 15:02:19.889837 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:19.891572 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:19.893307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 15:02:19.893307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-9a757c5d-75cb-49b3-865e-a9dfa27ef145 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 404 len: 111 microversion: 2.1 time: 0.040035 Oct 05 15:02:19.893307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 425/736] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:02:19 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 15:02:19.899171 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-eaa07200-5bcd-4be1-a179-c24cfe1f71e4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '.version_select at 0x7f77c95c5c10>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:19.925260 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-eaa07200-5bcd-4be1-a179-c24cfe1f71e4 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1971162760" status: 202 len: 0 microversion: 2.1 time: 0.032001 Oct 05 15:02:19.925852 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 312/737] 10.222.0.23 () {58 vars in 1221 bytes} [Wed Oct 5 15:02:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1971162760 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 05 15:02:23.186413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:23.186941 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:23.189339 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:23.195166 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:23.195642 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:23.196071 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:23.196758 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:23.197202 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:23.197671 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:23.201588 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG dbcounter [-] [120605] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:23.215098 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d HTTP/1.1 Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 15:02:23.216672 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.218393 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': 'ad3ee7bf-c734-421f-a660-1cc54792201d'} Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 15:02:23.219307 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.220846 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.222449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 15:02:23.222449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-ef54d3c5-aee5-4620-8754-83fd8e9ea59f tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 404 len: 111 microversion: 2.1 time: 0.037911 Oct 05 15:02:23.222449 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 426/738] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 15:02:23 2022] DELETE /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 15:02:23.227566 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:23.228170 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: ad3ee7bf-c734-421f-a660-1cc54792201d] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:23.235284 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:23.235762 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:23.236191 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:23.236879 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:23.237318 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:23.237769 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:23.251601 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d HTTP/1.1 Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 15:02:23.253235 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.255037 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': 'ad3ee7bf-c734-421f-a660-1cc54792201d'} Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 15:02:23.256159 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. Oct 05 15:02:23.257794 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:23.259275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance ad3ee7bf-c734-421f-a660-1cc54792201d could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 15:02:23.259275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-b02a308d-c308-42d4-b920-18771896b214 tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d" status: 404 len: 111 microversion: 2.1 time: 0.031635 Oct 05 15:02:23.259275 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 313/739] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:02:23 2022] GET /compute/v2.1/servers/ad3ee7bf-c734-421f-a660-1cc54792201d => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 15:02:24.405014 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:24.405591 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.compute.api [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120605) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:24.413332 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:24.413851 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:24.414285 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:24.415019 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:24.415467 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:24.415881 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG oslo_concurrency.lockutils [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120605) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:24.438404 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.wsgi [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: request: DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 HTTP/1.1 Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept: application/json Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Accept-Encoding: identity Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Connection: close Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Length: 0 Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Content-Type: application/json Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Host: 10.222.0.23 Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: Openstack-System-Scope: None Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: User-Agent: python-urllib3/1.26.12 Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Id: None Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Domain-Name: None Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Identity-Status: Confirmed Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Is-Admin-Project: True Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Id: default Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Domain-Name: Default Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Role: reader,member Oct 05 15:02:24.441091 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Roles: reader,member Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Id: default Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Domain-Name: Default Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: method: > Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.444270 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: action_args: {'id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7'} Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: {{(pid=120605) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 15:02:24.446128 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.449413 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.452048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: DEBUG nova.api.openstack.wsgi [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. {{(pid=120605) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 15:02:24.452048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: INFO nova.api.openstack.requestlog [None req-a6ec46b2-bbb4-4247-8646-136edcdde24d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 404 len: 111 microversion: 2.1 time: 0.042104 Oct 05 15:02:24.452048 dv-cinder-835878-4-iscsi devstack@n-api.service[120605]: [pid: 120605|app: 0|req: 427/740] 10.222.0.23 () {58 vars in 1239 bytes} [Wed Oct 5 15:02:24 2022] DELETE /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 05 15:02:24.455063 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Calling method '>' {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 05 15:02:24.455671 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.compute.api [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] [instance: 4b0af790-f4eb-48b1-b03e-06f2cb5668c7] Fetching instance by UUID {{(pid=120606) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 05 15:02:24.458145 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_api:SELECT=26,nova_api:DELETE=1 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:24.463600 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:24.464077 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:24.464506 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:24.465186 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Acquiring lock "003460f2-e2d0-447e-9352-d4e433e26efc" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 05 15:02:24.465666 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 05 15:02:24.466098 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG oslo_concurrency.lockutils [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Lock "003460f2-e2d0-447e-9352-d4e433e26efc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120606) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 05 15:02:24.469390 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG dbcounter [-] [120606] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:24.481462 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.wsgi [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] HTTP exception thrown: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Request method failure captured: Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: request: GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 HTTP/1.1 Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept: application/json Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Accept-Encoding: identity Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Connection: close Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Length: 0 Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Content-Type: application/json Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Host: 10.222.0.23 Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: Openstack-System-Scope: None Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: User-Agent: python-urllib3/1.26.12 Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Auth-Token: gAAAAABjPZdQtfvIu7RDCvmquyoe6oTRSd3-0_pcIqoXQ8VqZu2LcoU8lp4soyI17FreTv0ASauN-ZXyaIE6c-ZVMqV06esVm09VSaPd2MavAPzK32QdPqtW_sJW3aVWfC3SfjUAaDhVt4gsNeq4eQIG-XwqDbkbdu91sB_dDwOSKn7veWJSH-Y Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Id: None Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Domain-Name: None Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Identity-Status: Confirmed Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Is-Admin-Project: True Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Id: default Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Domain-Name: Default Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Project-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Role: reader,member Oct 05 15:02:24.483078 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Roles: reader,member Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Service-Catalog: [{"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/placement"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:8080/v1/AUTH_ff81348c32db4e40a8e331317d14e3d2", "adminURL": "http://10.222.0.23:8080"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/image"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23/volume/v3/ff81348c32db4e40a8e331317d14e3d2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.23:9696/networking"}]}] Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Id: ff81348c32db4e40a8e331317d14e3d2 Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1573353001 Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Id: default Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Domain-Name: Default Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Id: 7de0d01f891d4d73bc44810aacbca477 Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: X-User-Name: tempest-AttachVolumeTestJSON-1573353001-project Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: method: > Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: exception: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.484774 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: action_args: {'id': '4b0af790-f4eb-48b1-b03e-06f2cb5668c7'} Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: {{(pid=120606) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 05 15:02:24.485869 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. Oct 05 15:02:24.487412 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: ERROR nova.api.openstack.wsgi Oct 05 15:02:24.488872 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: DEBUG nova.api.openstack.wsgi [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] Returning 404 to user: Instance 4b0af790-f4eb-48b1-b03e-06f2cb5668c7 could not be found. {{(pid=120606) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 05 15:02:24.488872 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: INFO nova.api.openstack.requestlog [None req-dea33fe9-9ace-4a42-a31a-309fdcb44d9d tempest-AttachVolumeTestJSON-1573353001 tempest-AttachVolumeTestJSON-1573353001-project] 10.222.0.23 "GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7" status: 404 len: 111 microversion: 2.1 time: 0.033721 Oct 05 15:02:24.488872 dv-cinder-835878-4-iscsi devstack@n-api.service[120606]: [pid: 120606|app: 0|req: 314/741] 10.222.0.23 () {58 vars in 1236 bytes} [Wed Oct 5 15:02:24 2022] GET /compute/v2.1/servers/4b0af790-f4eb-48b1-b03e-06f2cb5668c7 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)